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ABSTRACT , 

This Report describes 



presents its resuiL:,, omu — - _ ■ . 

contracting as a means for improving education. Participants were students 
grades 7-9 in one junior high school and from grades 4-6 i n one elementa.y 
Lhool; both schools were in the inner city and had an almost entirely black 
student population. Learning Research Associates (LRA; provided the training 
and materials for conducting an individualized program of ^ 

to the program teachers. Individualization consisted primarily of dete.mmmg 
each student's strengths and weaknesses in reading by means of a diagnostic tes., 
and assigning the materials that would remedy the weaknesses uncovered. Despite 
the improvement In classroom atmosphere, the results of the final tes i ng s 
that Students at the Junior high level gained in reading achievement on y about 
as mulh as they had bten gaining in the past and that students at the elementary 
level were, in many instances, scoring lower °ri_post-tests than on pre tes . 
Interim tests of performance on assigned objectives demonstrated that the _ 

a=?ually reacLd the program objective,, bdt there i, no way to prove that the 
program was reaponaible for these results. The Norfolk program _ 

-performance contracting doas not autoi^tically solve the deeply rooted btoblems 
Of compensatory education. For related documents, see ED 056 24?. 249, 2p0, 2b 1, 
and 252. (Author/CK) 
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PREFACE 




This Report is a product of Rand’s study of performance contracting in educa- 
tion, The study is soonsored by the Assistant Secretary for Planning and Evaluation, 
U.S. Department of Health, Education .and Welfare, under Contract No. HEW-OS- 
70-156. 

Case Studies in Educational Performance Contracting comprises six volumes. 
Each is a self-contained study; together they provide a multifaceted view of perform- 
ance contracting. The six volumes are: 

1. R-900/1-HEW, Conclusions and Implications, by P. Carpenter and 
G. R. Hall 

2. R-900/2-HEW, Norfolk, Virginia, by P. Carpenter 

3. R-900/3-HEW, Texarkana, Arkansas and Liberty-Eylau, Texas, by 
P, Carpenter, A. W. Chalfant, and G. R. Hall 

4. R-900/ 4-HEW, Gary, Indiana, by G. R. Hall and M. L. Rapp 

5. R-900/5-HEW, Gilroy, California, by M. L. Rapp and G. R. Hall 

6. R-900/6-HEW, Grand Rapids, Michigan, by G. C. Sumner 

This study is the second of three Rand Reports on the subject. The first Report 
was J. P. Stucker and G. R. Hall, The Performance Contracting Concept in Education, 
The Rand Corporation, R-699/1-HEW, May 1971, The third Report will be a per- 
formance contracting guide intended for use by educational officials. 
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SUMMARY 



For over ten years, national attention has been drawn to the State of Virginia s 
efforts to restructure public education to meet the expectations of the Federal Gov- 
ernment with regard to school integration. Although these efforts initially focused 
on black students, there was a growing awareness that there were both black and 
white students throughout the state who were educationally disadvantaged. A few 
years ago, the Virginia State Department of Education engaged the Bureau of 
Research in the School of Education at the University of Virginia to undertake an 
extensive study of the educational needs of Virginia’s public school students. This 
study highlighted the educational deficiencies of students not only in the inner cities 
but in the rural regions of the state. 

The availability of Federal funds for compensatory education through Title I of 
the Elementary and Secondary Education Act, coupled with the rising popularity 
of performance contracting as a possible route to improved student iichievement, 
encouraged the Virginia State Department of Education to embark upon an exten- 
sive jierformance contracting program fbr the 1970-71 school year. The program 
ultimately funded was directed at improving the reading skills of Title I students 
in seven Virginia school divisions— -three in the Appalachian region, three in rural 
south-central Virginia, and the Norfolk City Schools. The program was evaluated 
by the Bureau of Research at the University of Virginia; Learning Research Associ- 
ates (LRA), a subsidiary of Universal Education, Incorporated, provided the instruc- 
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tionai elements of the program, and Education Turnkey Systems had a contract to 
supply management support to the State Department of Education and the school 
divisions. 

This Report describes the program in Norfolk, presents its results, and draws 
some inferences from the Norfolk experience for the utility of performance contract- 
ing as a means for improving education. 

The students selected to participate in the Norfolk program were from grades 
7-9 in one junior high school and from grades 4-6 in one elementary school. These 
schools were in the inner city smd had an almost entirely black student population; 
the faculties were racially mixed. Althou.gh the students in the program were 
academically among the poorest in the city, there were other inner city schools 
whose students wet e doing at least as poorly. In particular, students at one junior 
high school who were in the regular remedial reading program, funded under Title 
I, were even poorer readers than were junior high students in the program. 

LRA provided the training and materials for conducting an individualized pro- 
gram of instruction in reading to the program teachers, who were former Title I 
reading teachers. The individualization consisted primarily of determining each 
student’s strengths and weaknesses in reading by means of a diagnostic test, and 
assigning the materials that would remedy the weaknesses uncovered. A set of 
objectives in word attack, vocabulary, and reading comprehension had been devised, 
along with a rich variety of materials keyed to the objectives. The emphasis was on 
teaching basic reading skills; reading comprehension was not as heavily stressed, 
especially at the elementary level, because of the severe deficiencies of the students, 
some of whom were essentially illiterate. 

Observations of student activities in the program classrooms in the late spring, 
compared with observations of regular classroom activities, verified that the teach- 
ers had succeeded in al„ering the method of instruction to a surprising degree. 
Students managed their own work with little difficulty and gave evidence of enjoying 
what they were doing. Regular classroom teachers believed that program students 
had become more interested in reading and had gained competence. Many teachers 
were envious of the wealth of materials to be found in LEA’s reading centers (al- 
though the regular remedial reading rooms were fully as well endowed). 

Despite the evident improvement in classroom atmosphere, however, the re- 
sults of the final testing were disappointing. At the junior high level, students gained 
in reading achievement only about as much as they had been gaining in the past. 
What was worse, the post-test scores of students at the elementary level were, in 



t 



B 



I 

i 



5 



many instances, even lower than the pre-test scores. - he reason for this unhappy 
result may have been a mismatch between the content of the instructional program 
and the content of the standardized tests used to measure its efFectiveness. The 
heavy emphasis on word-attack skills was a natural first step in remedying severe 
reading deficiencies, but tests of word-attack skills, which must be administered 
orally, were not included in the standardized tests used because of difficulties of 
administration, 

That the elementary students actually had reached the program objectives was 
demonstrated by a sampling of student mastery of these objectives in interi u tests 
of performance on assigned objectives. These tests indicated that students had mas- 
tered an average of over 8S percent of the objectives assigned them. However, 
because there was no pre-test of these objectives, there is no way to prove that the 
program was responsible for these results. 

The program used an indepeiident evaluator to assure the integrity of test 
results, but also thereby thwarted the full use of evaluative data. Program personnel, 
did not participate in the selection and administration of tests, so that the mismatch 
betvreen test and program content at the elementary level was not readily apparent 
An interim evaluation report might have pinpointed this difficulty as well as other 
problems of an administrative nature. 

The actual cost of the program to LRA., Education Turnkey Systems, and the 
Bureau of Research at the University of Virginia was not divulged to Rand, What 
it would cost the Norfolk City Schools to implement the program as part of their 
regular compensatory curricula is of greater interest, however. A comparison of a 
rough cstim.ate of this cost with a similar estimate of the cost of the existing r.-'medial 
reading program funded under Title I suggests that LRA’s approach would cost some 
25 percent less. Since the tvra programs are probably of at least equal effectiveness 
(or will be if the content balance at the elementary level is improved), the LRA 
program appears to be a promising alternative for Norfolk, Two steps are now being 
taken that may correct the deficiencies of the 1970-71 program: restructuring the 
content at the elementary level for a better balance between word-attack skills and 
reading comprehension, and the use of a more interactive evaluation that will 
contribute directly to the improvement of the program as it develops. 

The Norfolk program demonstrated that performance contracting does not au- 
tomatically solve the deeply rooted problems of compensatory education. In fact, the 
mechanism of performance contracting can introduce difficulties in the development 
of effective instructional programs: in Norfolk, the need to maintain test security 
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virtually denied the use of evaluative data for program improvement during the 
crucial implementation phase. Tying contractor payment to students’ scores on tests 
also highlighted the problems of test selection and administration. 

There were some positive features, however, and they were sufficiently attrac- 
tive lo encourage the Norfoll? City Schools to expand the LRA approach during the 
1971-72 school year. First, the performance contract brought talent from outside of 
the school system to bear on a serious educational problem. A considerable amount 
of time and money had already gone into the development of the LRA system — 
money that came from publishers and others in the education industry. The fruits 
of this investment were made available to Norfolk. 

Perhaps more important, outsiders working under a guarantee of reward for 
good performance seemed to be able to operate more freely than could people with 
established roles in the school environment. The improved emotional atmosphere 
within the reading centers probably had a strong impact on everyone who observed 
the program. This atmosphere was largely a result of LRA ’s emphasis on student 
self-direction and was implemented in the reading centers in a surprisingly short 
time. 

In sum, radical departures from customary practice were adopted and are being 
extended in Norfolk’s target schools. 
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I. INTRODUCTION 




This Report describes the performance contracting program in Norfolk, Vir- 
ginia, during the 1970-71 school year. Norfolk was one of seven school divisions* 
participating in a performance contracting program for teaching reading, sponsored 
by the Virginia State Department of Education. The other participants — school 
divisions in Wise County, Dickenson County, and Buchanan County (often referred 
to as "Southwest”), and in Prince Edward County, Mecklenburg County, and Lunen- 
burg County (often referred to as "Southside”) — are shown in Fig. 1. These divisions 
were chosen by the State Department of Education to provide a representative 
sampling of the state’s Title I schools. The program students in Norfolk and South- 
side were primarily black, those in Southwest primarily white. 

Besides the State Department of Education and the school divisions, the Bureau 
of Research in the School of Education at the University of Virginia at Charlottes- 
ville participated in the program as the independent evaluator. Education Turnkey 
Systems contracted to p-ovide management support, and Learning Research Associ- 
ates, a subsidiary of Universal Education, Incorporated, provided the instructional 
elements. 

This Report begins with descriptions of Norfolk and the Norfolk School Divi- 
sion, It then describes the program schools and their surrounding neighborhoods, 
and regular classrooms in the program schools and in the remedial reading program 

' In Virginia, a school division is equivalent to a school district In other statm. 
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Fig. 1— Locati-m of Counties and Schools in Virginia Program 



funded under Title I. The latter description provides a basis for comparison with the 
contract reading centers. 

The program plan is then analyzed and the parties to the contract (other than 
tlie Norfolk School Division) are described. The history of the program as it actually 
developed is then treated in some detail, concluding with an assessment of the 
outcomes of the program as well as its costs. 

A final section presents some conclusions as to the impacts of the program on 
the Norfolk School Division. 
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II. THE NORFOLK SCHOOL DIVISION 



Norfolk is an Atlantic seaport in the southeastern corner of Virginia (see Fig. 
1), In 1970, Norfolk’s population was about 300,000 with about 28 percent black.^ 
Its outstanding harbors make it an ideal port and naval base, and most of the key 
installations of the U.S. Navy’s Atlantic Fleet are in the area. The port and naval 
installations provide major support for the area’s economy. The Federal Govern- 
ment employed 17 percent of working civilians in the Norfolk-Portsmouth Standard 
Metropolitan Statistical Area in June 1971.® This results in a relatively stable 
economy and moderate rate of unemployment compared with other cities of similar 
size. In June 1971, 6.1 percent of the Norfolk-Portsmouth civilian labor force was 
unemployed. 

Even so, about 30 percent of Norfolk’s families (not including single-person 
families) had an annual income of less than $3000 in 1965; thus, there was a sizable 
population below the poverty line. In addition, of persons 25 years old or older, about 
27 percent had less than an eighth-grade education in 1966.* As with other cities in 
its class, aggregate data characterizing Norfolk are misleading, since they tend to 



“ U.S. Department of Commerce, Bufeau of the Census, I9W Census of Population, General Popula- 
tion Characteristics, Advance Report PO(V2)-47. Virginia, 1971. 

* Trends in Employment Hours and Earnings, Virginia and Standard Metropolitan A nas, Vol, 21, 
No. 7, Division of Research and Statistics, Virginia Department of Labor and Industry in cooperation with 
U.S. Bureau of Labor Statistics, Richmond, Virginia, July 1971. 

* Neighborhood Analysis, Norfolk City Planning Commission, August 1SS7. 



understate the magnitude of the problems that exist in the inner city areas. These 
problems are discussed at greater length in Sec. III. 

The Norfolk School District comprises three senior high schools including 
grade.s 9 through 12, two senior high schools including grades 10 through 12, ten 
junior high schools including grades 7 through 9, and 61 elementary schools includ- 
ing various combinations of grades 1 through 6. Total "membership”'^ as of the end 
of January 1971 was about 54,000 students, with about 23,000 in secondary schools 
and 31,000 in elementary schools. Total enrollment for the 1970-71 school year 
(students who have entered school during the school year) was about 59,000 students 
to that date, indicating a net loss of about 5,000 students during the school year. 
Table 1 presents statistics on student enrollment provided by the central adminis- 
tration of the school division. 

The central administration of the Norfolk School Division controls the b'adget, 
the hiring and firing of teachers, and the specifications of curriculum. Budget re- 
quests are submitted by building principals, and in general, money is allocated on 
a per-pupil or a per-teacher basis, Building principals, of course, exercise some 
degree of control over the teaching styles of classroom teachers, depending on their 
personal ways of running their schools. Thus, a fair degree of autonomy can exist 
at several levels in the system even though the cen il administration holds the 
purse and personnel strings. ' ' 

Norfolk spends about $41 million a year on its public schools, or about $750 per 
student. Of this, $20 million is supplied from the city, $13 million from the state, 
and $8 million from the Federal Government,® The Federal Government supplies 
considerably more support (about 19 percent of the total) for public education in 
Norfolk than the national average of around 7 percent, primarily because Norfolk 
receives about $4 million as a. Federally impacted area. Since the per-pupil cost for 
Virginia as a whole is around $700, one might conclude that Norfolk is contributing 
relatively less to its public schools in light of its ability to pay. Norfolk ranked 26th 
out of 36 Virginia cities in terms of equivalent true tax rate in 1968.’ 

The direct avenues for community control of the schools that exist in other 
big-city school districts seem to be lacking in Norfolk. Citizens cannot express their 

° Students who aro actually In school at the time the roll Is taken or who are presumBd to be in school 
and not to have dropped from the rolls. 

® Financiai and Statistical Data, Norfolk City Schools, December 1970. 

’ Equivalent true tax rate = (local expenditures) + (wealth per child based on values of real estate 
and puUic service corporations). .Facing Dp: Sialislical Data on Virginia 's Public Schools, State Depart- 
ment of Education, Richmond, Virginia, January 1970. 
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Table 1 

SUMMARY OF ENROLLMENT AND MEMBERSHIP REPORT 
OF SCHOOL BOARD OP THE CITY OF NORFOLK, 
JANUARY 31, 1971 



Grade 


Membership on Roll , January 31 


Total 

Membership 

1970-71 


Secondary 


Elementary 


1-3 




15,267 


15,267 


4-6 


— 


14,035 


14,035 


7 


3,498 


1,276 


4,774 


8 


4,543 


— 


4,543 


9 


4,195 


— 


4,195 


10 


3,705 




3,705 


11 


3,486 


— 


3.486 


12 


2,801 


— 


2,801 


S.E.® 


300 


672 


972 


Total 


22,528 


31,250 


53,778 





Enrollment 


A.D.A.b 


School Level 


1970-71 


1970-71 


Secondary 


24,384 


19,760 


Elementary 


34,408 


28,721 


Total 


58,792 


48,481 



Special education. 
^Average daily attendance. 
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views through school bond Elections or votes on changes in basic tax rates. Instead, 
the schools depend for their financing on the city council, through which their 
budget goes and from which city taxes are meted out. The citj' council sets the tax 
rate and is accountable to the citizens c .ily through the election of council members. 
The council finances buildings and operations through bonds floated within their 
existing bonded indebtedness. Even Federal funds for public education must at least 
formally pass through the city council. 

The school board is appointed by the city council rather than elected by the 
citizens of Norfolk; therefore, citizens cannot directly influence its composition. The 
board only sets general policy and leaves the administration of the schools to the 
staff of the school division. The Education Association of Norfolk, the only group 
representing Norfolk teachers, works directly with the board in support of teachers’ 
interests. There is apparently no teachers’ group expressing independent opposition 
to school board policies as is found in cities with teachers’ unions. 

The foregoing may partly reflect a general lack of concern with the public 
schools in the community at large, a conclusion supported by opinions expressed by 
a dozen or so school and community leaders, building principals, and teachers, In 
the past, students whose families could afford it attended private schools; families 
of students in the public schools were less involved with the content and quality of 
education than with other aspects of schooling. The most salient example is the 
intense concern of the community with the continuing process of school integration, 
which has radically altered the school system during the past year. In 1959, the 
Norfolk City Schools were closed in compliance with the State of Virginia’s "massive 
resistance” efforts toward school integration. The case has been in court ever since, 
but progress in integration was slow. 

As of April 1970: 

All secondary schools in the city have racially mixed student bodies. The 
longer range plan calls for making the racial balance in secondary schools 
the same as that in the rest of the city — about 70% white and 30% black. . 
Thirty-five out of 54 elementary schools have racially mixed student bodies. 
Student bodies are 100% black or white only where reasonable school dis- 
tricts cannot be drawn to include housing occupied by both races.* 

“ Model Cities' Second Year Plan, Model Cities Agency, U.S. Department of Housing and Urban 
Development, Norfolk, Virginia, April 1970. 



Before the beginning of the 1970-71 academic year, school district boundaries 
were redrawn, students were to be bussed across district boundaries, and faculties 
in all schools were altered to attain a racially mixed staff. At Campostella, (one of 
the comparison schools in the performance contracting program) only six former 
teachers remained out of a staff of 25. Bussing of students was compulsory, and 
parents had to pay $5 a month for each child bussed. This worked an obvious 
hardship on poor families with large numbers of children, It also caused considera- 
ble disruption in some schools (apparently depending in large part on the leadership 
in particular schools) and in some cases resulted in the departure of white students 
from the Norfolk public schools to private schools or elsewhere. After a relatively 
stable membership that peaked at around 56,000 students in 1969-70 (see Table 2), 



Table 2 

TRENDS IN MEtfflERSHIP, NORFOLK CITY SCHOOLS 



School Level 


1965-66^ 


1956-67® 


1967-68® 


1968-69® 


1969-70^’ 


1970-71^ 


Total Secondary 
Total Elementary 


20,511 

3.3,496 


20,102 

33,448 


21,134 

32,893 


21,683 

32,393 


23,444 

32,398 


22,538 

31,250 


Total 


54,007 


53,550 


54,027 


54,076 


55,842 


53,778 



Membership as a£ June of academic year. Taken from Enrollment wd, 
Menbership RepOTt, School Board of the City of Norfolk (periodical) . 

*^Memberahlp as of January of academic year. 
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1970-71 saw a drop of about 2000 students, "probably 70 percent of whom are white," 
according to one offlcial. That desegregation may be having an effect on the number 
of whites with school-age children in Norfolk is also suggested by the recent census 
figures, which give the Standard Metropolitan Statistical Area (SMSA) of Norfolk 
a total black population of about 28 percent, but indicate that nearly 39 percent of 
the population in the 5 to 14 age group is black.® The fact that the total black 




^ Bureau of the Census, op. cit. 



population of the Norfolk-Portsmouth area is about 25 percent, while only 23 per- 
cent of the population in the 5 to 14 age group is black, indicates that the former 
figures do not simply reflect a propensity of blacks to larger families. 

Norfolk provides kindergarten only under Federal funding; instruction in read- 
ing (except as part of language arts) in the later elementary grades (4 through 6) is 
also provided solely by Federal funds under Title I of the ESEA and Model Cities. 
Students are usually not held back more than two years in a single grade, although 
"social promotions” are not explicitly supported by division policy. It is not unusual, 
however, to find students who are relatively older than their peers in the upper 
grades. For example, nearly 55 percent of the eighth-grade students in the reading 
program at Campostella Junior High were one or more years older than the normal 
14. The minimum age at which a student may legally leave school in Virginia is 17. 

There are still remnants of the former Norfolk tracking system in which the 
guidance counselors placed students in either regular, "modified” (i.e., below aver- 
age), or accelerated classes. The “modified” track has been abolished in some schools 
on the basiis that it tended to perpetuate segregation. 

After the third grade, students study language arts (based on Roberts’ series of 
English texts), arithmetic, social studies, and science, with additional time for physi- 
cal education, music, and other activities. Beginning in the seventh grade, students 
can choose from a wide variety of electives such as home economics, Spanish or 
French, metalwork, electrical work, woodwork, business, mechanical drawing, art, 
or band. Remedial reading programs under Title I are available in all Title I schools, 
but are provided to relatively few students. In addition, a program for "teaching 
reading in the content areas” is provided at the target junior and senior high level, 
also under Federal financing. This program trains subject-matter teachers to use 
their subject as vehicles for teaching reading; reading is not taught as such, however. 
It is not too surprising, therefore, that many students, particularly those blacks whc 
do not use so-called “standard English” in the home, remain functionally illiterate 
throughout their progress through school. For example, the median score achieved 
by 30 percent of the lowest-scoring seventh-graders on the comprehension section 
of the Iowa Silent Reading test given in March 1970’° was equivalent to grade 4.1 
in terms of national norms. 

Grvup Testing Annual Report^ 1969‘70, Norfolk City Schools, August 1970= 
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III. SCHOOLS IN THE PERFORMANCE CONTRACTING 

PROGRAM 



Four Norfolk schools participated in the performance contracting program — 
two elementary and two junior high. All four schools are in inner-city neighbor- 
hoods, shown in Figs. 2 and 3. At the elementary level, students in grades 4, 5, and 
6 at St. Helena formed the treatment group, while students in the same grades at 
Young Park were used for a comparison group; students in grades 7, 8, and 9 at Jacox 
Junior High were selected for the treatment group, while the comparison group was 
drawn from the seventh grade at Campostella Junior High, 

The socioeconomic conditions in the neighborhoods surrounding the four 
schools are quite similar, although both of the comparison schools (Y oung Park in 
particular) draw from populations of somewhat poorer status than the treatment 
schools. This conclusion is supported by data presented in the Neighborhood Analy- 
sis** and summarized in Table 3. (Computations for Table 3 are displayed in the 
Table of Appendix D.) Only those Planning Districts*® that are predominantly black 
were used because the program students were black, writh only a few exceptions. A 
school survey in 1970 provided percentages of students from "low-income” (< $3000?) 
families as follows: St. 'Helena (treatment) 57 percent; Young Park (comparison) 69 
percent; Jacox (treatment) 54 percent; Campostella (comparison) 71 percent. Thus, 

“ Norfolk City Planning Cortitnission, op. cit. 

Geographical divisions used by the Norfolk City Planning CommiBsion (sea Pigs. 2 and 3), 
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SOURCE: CoMputed from data in Seighborhood Analysis. See table in Aj,(jendix D. 



both of the comparison schools drew from lower-level income families than did the 
treatment schools. 

In writing about the Model Neighborhoods Area, in which the program schools 
are found, Model Cities personnel had this to say:‘“ 

The , , , area is populated with groups of people who have traditionally 
accounted for the lowest family incomes in our society — 'blacks, elderly fami- 
lies, families on welfare, f lale heads of households. Each of the neighbor- 
hoods . . . has a high black to white ratio The current average formal 
education level ... is two grades below the city average, and the functional 
or actual educational gap is estimated to be much wider. 

Thus, even the fact that over half of the adult population in 1966 has had less than 
an eighth grade education does not truly reflect the ability of the residents to 
compete in the job market, because as noted before, students who literally cannot 
read are common in the later elementary grades. Their progress through school is 
justified by their ability to remember material taught orally. 

Much of the above situation is generational and is rooted both in past 
discrimination patterns and the results of those patterns and other factors 
working over a long period of time. The general nature of the local employ- 
ment market and the residents’ abilities to compete in that market are 
examples. The regional economy is wide in terms of variety of available job 
opportunities but it is not very deep — there are always jobs available, but 
not in great quantities as in regions dominated by heavier industry, and the 
jobs that do become available in any quantities at all are either in the "upper 
reaches,” requiring significant amounts of education and/or experience, or 
at the other end of the scale, requiring little education, skill or work experi- 
ence and offering little psychological or cash compensations. 

. . . student achievement levels as measured by a variety of standard 
testing procedures are low in comparison with achievement levels of stu- 
dents in other schools around the city . . . The proportion of Model Neighbor- 
hood Area high school students going on to higher education is estimated at 
about 1/4, while almost 1/2 of studente from schools in the rest of the city 
do so . . . 



” Model Cities Agency, op. cit. 

One hundred percent in some neighborhoods. 
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Norfolk households increased by 20,250 from 1950 to 1968, but again a 
significant drop in the rate of increase is evident, as 13,600 occupied housing 
units were added during the 1950’s versus 6,650 during the 1960s . . .As in 
the Model Neighborhood Area, Norfolk’s slowdown has been accompanied 
by a continuing racial shift in housing occupancy. White household in- 
creases in Norfolk during 1950-68 outgained Negro household increases 
(10,900 to 9,750), but almost all of the white increases (10,400) occurred 
during the 1950s while two-thirds of the Negro gains (6,150) occurred during 
tVie 1960s. 

From the time [of] the first public school integration in 1964 until 1968, 
vacancy levels in the white occupied housing inventory have been above 
normal and housing occupancy shifts from white to black occurred at a 
record rate in Norfolk. The fastest changing Neighborhood Areas contained 
the worst housing in terms of age and conditions, were in close proximity to 
redevelopment activities and were the easiest accessible to low- and moder- 
ate-income families facing a shortage of standard units. Housing areas with 
small changes in racial occupancy were those located away from inner city 
concentrations of ill-housed low-income negroes and which had public school 
integration of less than 30 percent initially.*® 

Except for Jacox Junior High, the program schools were relatively unaffected 
by the 1970-71 school year desegregation plans. The student bodies of both St. 
Helena and Campostella were all black, although the facilities were mixed. Young 
Park Elementary is in the middle of a housing project, and its students were also 
all black, with mixed faculty. Jacox, however, is near the edge of the black area 
downtown and was supposed to draw white students from an area nearby (see Pig. 
3). The expected total enrollment for Jacox was to be 1300 students, 800 of whom 
were to be white. Instead, only about 300 of the whites to be transferred to Jacox 
actually enrolled, and there was trouble at the school throughout the year. Extracur- 
ricular activities were reduced in order to minimize opportunities for trouble, the 
PTA was disbanded, student dances were forgone, and recess periods stopped. School 
and community leaders did not, however, take the case of Jacox as proof that 
integration would not work in Norfolk, because there was little trouble at Booker 
T. Washington, an integrated senior high in the Jacox area. In fact, some of the same 



parents who had children at Jacox strongly supported Booker T. Washington. 



“ Model Cities Agency, op. cit. 
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Discipline was a minor problem a' both of the elementary schools, and most 
troubles were not of a serious nature. This was not true at the junior high level, 
where suspensions were common. Although records of referrals and suspensions are 
kept, the information is extremely difficult to retrieve, being filed alphabetically in 
special folders. Rand personnel discussed the records for some of the treatment 
students at Jacox with the Assistant Principal, but he (and we) felt that it was not 
possible to determine whether the apparently improved attitude of a few of the 
students was attributable to the program or to social maturation. 

Some characteristics of students in the program schools are summarized in 
Table 4. Both of the treatment schools are considerably smaller than the schools 
from which comparison groups were drawn. In fact, St. Helena is so small that 
essentially the entire fourth, fifth, and sixth grades were needed to supply the 125 
students for the program. At Jacox, however, only about a seventh of the students 
were in the program. 

The table demonstrates that the students in both the comparison and treatment 
schools are different in several ways from the average for Norfolk. Absence rates are 
higher at both the junior high and elementary levels, and IQs and reading scores 
are lower. In fact, the reading scores for St. Helena and Young Park may be decep- 
tively high, since the lowest score attainable on the SRA test is 3.1?^ The lower net 
loss for the program schools appears anomalous at first hand; however, this may be 
a reflection of the lower mobility for the black population than for Norfolk as a 
whole. 

The . , , brief profile of a predominantly low-income, black population with 
a significant proportion of elderly households concentrated in one area is 
important in terms of mobility — i.e„ the population’s continued occupancy 
of an area of marginal living conditions vs the abilities of groups within it 
to move somewhere else. Mobility rates developed by Hammer, Greene, Siler 
and Associates by race and income levels indicate that as a whole, blacks are 
less mobile than their white counterparts and that low-income blacks are 
least mobile of all. Model City blacks fall into the lowest mobility categories 
—the significant number of elderly households adds to the general immobil- 
ity of the low-income population, since their needs are fairly static and 
habits set.*® 



Science Research Associates, Multi-Tjevel T^t, Form C, Blue Level Teacher^s Handbook, science 
Reiaarch Asiociates, Chicagg^ 111.) 60611. 

*■ Model Citiei Agency, op. cit. 
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Graup Testing Aaimal Eeport, 1X9-70, N’orfolk City Schools, August 1970. 



The most striking difference between the treatment and comparison schools is the 
very low loss rate at Young Park. This is probably because most of these students 
live in the Young Park public housing development, which may discourage mobility. 

The classroom teachers at the program schools are mostly white, married 
women who have from 5 to 10 years of teaching .experience. Those interviewed all 
had bachelor's degrees, by and large earned at nearby colleges and universities such 
as Old Dominion University in Norfolk or Virginia State College in Petersburg. Most 
of the teachers were also engaged in furthering their education, frequently by taking 
graduate-level courses at Old Dominion University. 

Both of the building principals for Jacox (treatment) and Campostella (compari- 
son) junior highs are male and black; St. Helena’s (treatment) is male and white; 
Young Park’s (comparison) principal is female and black. This small sample is 
indicat. /e of the extent of desegregation in Norfolk, at least within the staff of the 
school division. 

The regular (nontreatment) classes observed at the elementary level at St. 
Helena and Young Park contained about 25 students. The classroom teacher taught 
the same group of students for about 6 hours, with breaks for rest and lunch and 
with some relatively free time in the classroom during which other teachers con- 
ducted special aciivities such as music or physical education. Teachers sliared aides 
who assisted them by grading papers and performing other services. The teaching 
method most frequently observed was teacher-directed class discussion and oral 
drill, sometimes focusing on a media presentation such as a filmstrip. Teachers 
seemed almost to avoid working with the printed page — possibly because of the 
mismatch between the materials provided and student reading capabilities. For 
example, the fifth grade used Roberts’ English for the fifth grade — far too so- 
phisticated for children with almost no reading skill. The rooms at St. Helena ap- 
peared to be better furnished and equipped than those at Young Park, which seemed 
to be an older or less well-maintained school. 

Teachers of academic subjects at the junior high level carried four classes with 
an average enrollment of about 30 students per class. In addition thsi' had a confer- 
ence period for planning their work, previewing films, and so on, and they had other 
assignments such as hall duty or serving as a lunchroom supervisor. The teaching 
method was again teacher-directed class discussion and oral drill. Teachers tried to 
individualize instruction to some extent by grouping students by ability level for 
work at their desks, but full individualization was impossible because of the range 
of abilities to be accommodated and the lack of materials and equipment to fit this 
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range. Even in "accelerated” classes at the junior high level, there were students 
who lacked basic reading skills. Some teachers supplemented regular texts with 
special materials in an effort to fill the gap between materials written for the 
students’ grade level and their true academic skills. 



IV. OTHER READING PROGRAMS IN NORFOLK 
TITLE i SCHOOLS 



In addition to the performance contracting program, Title I funds are being 
used to finance two other reading programs in the Norfolk schools. These will be 
described briefly because they are being compared with the performance contracting 
program. One is essentially a remedial reading program conducted at both the 
elementary and junior high levels. This is closest to the performance contracting 
program in intent because its objective is to improve student reading skills; there- 
fore the rempdial reading programs at Jacox and St. Helena were replaced by the 
performanre contracting program. 

The other reading program is aimed at teaching reading by stressing that skill 
in the jourse of regular classroom teaching of academic subjects such as social 
studios. Termed "teaching reading in the content areas," the program is provided 
at junior and senior high schools. Although this program is not as close in objective 
to the performance contracting program, it could be more effective in improving 
reading comprehensionhBcause of its stress on that skill. Jacox retained its program 
for teaching reading in the content areas. 

In general, classes in rem.adial reading are small — on the order of 10 to 12 
students per class — and contain children from several grades. Reading teachers 
carry a total load of abcut 60 students, somewhat less than half the number of 
students instructed by a teacher in the performance contracting program. At the 
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elementary level, the teacher handles six classes a day, four days a week, with one 
day for planning. Like other elementary teachers, she has a half-hour for lunch and 
short breaks between classes. During her planning day, other students who are 
having difficulties often come to her for help. At the junior high level, the reading 
teacher teaches five periods of reading and has one period for planning, during 
which time students „lso come for additional help. 

Rooms for remedial reading are considerably smaller than regular classrooms 
and are somewhat less formally furnished. Students often work around tables or in 
small groups. A large variety and number of books, workbooks, filmstrips, kits, and 
games are available, supplied by Title I funds. In one class, students were working 
with a Hoffman teaching machine. 

The teaching method used seems to be largely contingent on the individual 
teacher’s approach. One teacher used teacher-directed, small-group instruction al- 
most exclusively, drawing on only a few of the materials available, Another ar- 
ranged for each student to work relatively independently, drawing from a wide 
variety of materials as needed. Neither teacher had devised a system whereby each 
student could get and return materials as he needed them and could check his own 
progress. This is probably why the inore "individualized” classroom looked disor- 
derly and "inefficient,” in terms of the amount of time that each student was 
actually working, compared with the more structured class. 

The program for reading in the content areas is also funded by Title I and Model 
Cities. Its objective is to improve reading skills by training teachers to u.se subject- 
matter materials to this end. Originally, teachers were required to attend in-service 
training sessions during the school year, but during the 1970-7 1 year attendance was 
voluntary, apparently because some teachers resented being required to attend. The 
"reading content” program is being conducted in seven schools — two senior highs 
and five junior highs. In each of the senior highs there is a coordinator for the 
program and a reading content specialist for each of the major disciplines — social 
science, science, math, and English; in each of the junior highs, the coordinator 
works with only two reading content specialists. Each reading content specialist, 
who is a classroom teacher, work,= in turn with two regular teachers. 

The central administration has evaluated the performance of students who 
have teachers who are reading content specialists. Although the data are not availa- 
ble at this writing, the evaiuators expect that the Stanford Achievement Test** will 
show significant differences in gain over the November-to-May period between stu- 

Unfortunately, the SAT was not usfd to evaluate the performance contracting program. 
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dents who did and who did not have teachers who were reading content specialists. 
The evaluators also have administered a measure of student attitude, developed by 
Dr. Herber of Syracuse University, who has devised the reading content program. 
Ed Doughtrey, Program Supervisor, may be contacted for further information. Al- 
though it is hii-,hly desirable to determine whether a program has had any effect on 
si,.ident attitude, questionnaires of this type may reflect what the respondents think 
they should answer as well as what actually motivates their behavior. 
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V. PERFORMANCE CONTRACTING PROGRAM: 
INITIAL STAGES 



In the spring of 1970, Governor Linwood Holton and Dr. Woodrow W. Wilker- 
son, Superintendent of Education for the State of Virginia, were convinced by Pey- 
ton Cleveland, Washington-based consultant to the Governor’s office,^'’ that per- 
formance contracting would be desirable for Virginia. Dr, Wilkerson anu others in 
the State Department of Education saw this as an opportunity to follow through on 
some cf the recommendations put forth in the educational needs assessment study 
done for the state by the Bureau of Research in the School of Education at the 
University of Virginia, Charlottesville, Education Turnkey Systems (ETS) was al- 
ready in contact with Dr, Wilkerson’s advisors and offered advice and assistance 
from the outset. 

The State Department of Education obtained permission from the Office of 
Education to use administrative money under Title I to employ a management 
support group and an independent evaluator. At first, the State Department wanted 
the contractor to guarantee to achieve stated behavioral objectives in reading and 
math for students in the lowest-income counties in the state— Norfolk inner-city 
schools, Dickenson County, and Prince Edward Country. However, at the first meet- 

"Where It’s Happening,** Education Turnkey News, Vol, 1, No, 2, Education Turnkey Systems, Inc,, 
May 1970, 



ing, the superintendents of schools for these counties resisted diverting enough 
current Title I funds to pay for programs in both reading and math. In addition, ETS 
pointed out that no company would bid on previously stated behavioral objectives 
(probably because such objectives would be unlikely to correspond with those dealt 
with by any one company’s materials and techniques). Therefore, the program was 
cut back to reading only and extended to cover students in four more counties, and 
the behavioral objectives were left up to the contractor. Representatives from each 
of the seven school divisions attended the second meeting, after which the Request 
for Proposal (RFP) was written. 

About 40 bidders responded to the RFP by attending the bidder’s conference in 
July. At the conference, representatives from each of the school districts, including 
classroom teachers, and state administrators of Title I funds scored the proposals 
submitted, using criteria provided by ETS. The choice was almost unanimous for 
Learning Research Associates (LRA), who sent Alan Cohen as their representative. 
Cohen had developed a practical approach to the teaching of reading that appealed 
to the teachers involved and convinced them that LRA®* had the most to offer in the 
way of instruction. As usually happens, the bidders’ conference generated considera- 
ble rancor among the closest contenders and their supporters. 

The contracts for independent evaluation and management support were sole- 
source; that is, they w<sre awarded to a preselected contractor without competitive 
bid. The State Department of Education had worked with the Bureau of Research 
in the School of Education in the past and wanted to continue that relationship. Dr. 
Charle.s Woodbury of the Bureau wrote the first draft of the contract; then the State 
Department wrote the final version.*® The Bureau will receive $74,000 for its work 
(nearly half of the maximum that LRA could receive for the statewide program). The 
same general procedure was followed with HITS, but no copy of the contract has been 
supplied to us, and we do not know what ETS received. 



THE PARTIES TO THE CONTRACTS 

Lee Brown, president of LRA, was formerly a vice president of Science Research 
Associates (SRA), where he worked on the design of the now-famous SRA reading 

LRA’s proposal is attached as Appendix A. - 
** Attached as Appendix B. 





kits. He subsequently formed Learning Materials, Inc., to support the rewriting of 
the kit, and later formed the company which became LRA in 1968, a subsidiary of 
Universal Education Corporation. LRA attempts to improve public education at all 
levels by activities such as conducting workshops, working with school districts, and 
training administrators; there are 16 fulLtime staff members and numerous consult- 
ants. Among other things, l.RA worked with schools in Pittsburgh and Fort Lauder- 
dale and at the Nova school in Florida to develop programs of Individually Pre- 
scribed Instruction (IPI). Roy Stern, LRA’s Director of School Services, believes that 
performance contracts are an effective way to introduce change into the schools and 
that once a school system has successfully completed a performance contn cting 
program the contractor should withdraw from primary responsibility for classroom 
instruction. 

When the Virginia contract came up, Brown persuaded Cohen to join him on 
it to provide the teacher training and troubleshooting in the field. The LRA proposal 
includes statements concerning a subcontract to Urban Education, Inc., originally 
a group of educators (including Cohen) that evaluated Federally funded programs 
in the New York area. Cohen stated that there was never a formal contract with 
either (JEI or himself, however — that the work he and his students at Yeshiva 
University performed in Virginia was done directly for LRA. Perhaps at the time 
LRA’s proposal was written, someone in LRA felt a need to formalize the relation- 
ship between themselves and Cohen’s group. Stern has supplied most of the purely 
managerial contacts between LRA and the Virginia schools. 



THE CONTRACTS 

The evaluation contract specifies little in detail; instead, the details of the 
evaluation are defined by implication in the contract with LRA.^® Because the 
evaluation contract is so brief, it will not be discussed further here. 

Although a copy of the contract with ETS has not oeen made available, the 
State Department of Education informed us that ETS contracted to perform the 
following tasks; 

• Assist the State Department in conferring with local divisions. 

Attached as Appendix C. 
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. Write the RFP. 

• Provide a list of prospective bidders. 

• Help riegotiate the contract and conauct the prebidders’ conference. 

« Conduct the evaluation of the proposals. 

• Help with monitoring and data gathering. 

« Calculate the cost of the program.'^'* 

Most of these activities were pointed toward assisting the State Department of 
Education in monitoring the progress of the program throughout the state rather 
than toward assisting the school divisions themselves. 

The preliminary draft of the contract between LRA and the Norfolk Public 
Schools, drawn up by ETS in September 1970, contained most of the major features 
of the contract that was eventually signed (Appendix C). There were, however, some 
important additions and changes (in addition to legal language), which will be 
discussed as the main points of the contract are reviewed. 

LRA’s primary responsibilities, set forth in section 1.04, were as follows: to 
provide a reading program comprising objectives for student performance, diagnos- 
tic tests of student mastery of these objectives, self-instructional materials keyed to 
the objectives, and equipment needed to develop basic listening skills; to train 
teachers to manage the program; to ensure that the program was being operated as 
designed by supervising and assisting teachers as required; and to provide a "special 
environment" for the reading centers, which generally meant a somewhat more 
comfortable room than the regular classroom. This was particularly important in 
Jacox, where the reading center was air-conditioned and the regular classrooms 
were not. Teachers and other instructional personnel were to be drawn from the 
Norfolk schools and were to remain in Norfolk’s employ. The State Department of 
Education would have final say in the resolution of all disputes. 

Students were to be selected on the basis of "grade level deficiencies” (degree 
unspecified) in reading as determined by the pre-test administered by the independ- 
ent evaluator. Thus, student scores on reading tests were supposed o be only an 
initial screening device; however, although a few students scored at grade level or 
higher on the pre-test, none was dropped from the program. There was no specifica- 
tion for the provision of control groups. 

An approximate distribution of students among grades was also prescribed, 
which was not satisfied for St. Helena (see Table 6), as there were more treatment 



Not to be confused with the contractor payment. 



students in the fourth grade than in the sixth. The actual distribution at Jacox was 
more 'n line with the specification, partly because the turnover was so high in the 
ninth grade that many more ninth grade students participated in the program than 
remained throughout the full period of instruction. The rationale behind the spe- 
cified distribution was not clarified; the specified distribution for Jacox, however, 
was considerably different from that first set forth in the ETS draft, viz,, 100, 15, and 
10 students from the seventh, eighth, and ninth grades, respectively. 

Formulas for payment to LRA are presented in the contract and its amend- 
ment, The payment schedule was geared to results on standardized tests and on tests 
of "interim performance objectives.” On the basis of standardized pre- and post -tests: 

• LRA would be paid $63.75 for each student who gained 1.7 grade levels and 
received the fUll term of instruction. If less than 150 hours of instruction was made 
available by Norfolk, the required grade level gain (but not payment) would be 
reduced proportionately. This provision was not in the ETS draft. Since actual hours 
available turned out to be about 130, this reduced the guaranteed grade level gain 
to about 1,5. 

• LRA would be paid (or docked) $4.00 for each student who gained more (or 
less) than the guarantee and for each 0.1 grade level above (or below) the guarantee. 
However, in no event would LRA receive more than $15,937.50 for the 250 students 
(i.e., $63.75 per student) on the basis of the standardized tests. 

• If a student left the program early and had no post-test score, LRA would be 
paid a base amount equal to the amount paid for the average student completing 
the program (which would take account of penalty bonuses), but the amount would 
be reduced proportionately to the number of instructional periods actually com- 
pleted. Thus, LRA would be penalized (or rewarded) for this student if average 
full-term student achievement were below (or above) the guarantee. This proviso is 
considerably different from that in the ETS draft, which would have paid LRA for 
this student a flat rate of $37.50 per grads level gained by the average full-term 
student, reduced proportionately to the number of instructional periods actually 
completed, but with no bonus payments or penalties if the average gain were above 
or below the guarantee. 

m Specific provisions for students who entered late were to be similar to the 
foregoing; however, a special provision was made in the amendment that late- 
entering students who had not been pre-tested would be given the average pre-test 
score of their peers. 

• Provisions for payment for students with IQs below 75 were also added to the 
final contract; and had been omitted from the ETS draft. 
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On the basis of tests of interim performance objectives: 

« LRA would be paid $21.25 for each student who correctly answered 85 per- 
cent of the test items (later reduced to 80 percent, for ease of calcuation— see amend- 
ment, Appendix C, continued) on each interim objective prescribed for the student 
by LRA. There was no contractual specification of the minimum (or maximum) 
number of objectives that a child would be assigned. LRA would receive $2.00 more 
(or less) for every test item a student answered correctly beyond (or less than) those 
for the objectives originally prescribed for him, but in no event would LRA be paid 
more than $5,312.50 for the 250 students (i,e., $21.25 per s. adent) on this basis. Since 
the evaluator actually tested only a sample of the performance objectives for each 
student, as suggested in the amendment, it is possible that rewards or penalties were 
never meted out on the basis of the interim tests. 

♦ Rrovisions for payment for late-entering or early-leaving students were simi- 
lar to those for payment based on standardized tests. As before, the ETS preliminary 
draft omitted the reward/penalty feature for these students. 

Interim payments; 

• In any event, LRA was to be paid $17,000 in six equal installments to dofray 
the costs of the program as it progressed. This was 80 percent of the maximum 
amount LRA could expect to receive on the combined basis of the interim tests and 
standardized tests. 

The signing of the contract was delayed for two months by legal difficulties 
concerning the assumption of liability for students in the program. The ETS draft 
made Norfolk City Schools liable for student safety, which is not permitted under 
Virginia law. To resolve the problem, Norfolk City Schools finally paid LRA about 
$476 to obtain liability insurance. 

The contract describes testing procedures for determination of contractor pay- 
ment (validation). The major features of the validation plan were t lat the independ- 
ent evaluator would select, administer, and score one of at least three standardized 
achievement tests per grade to each child in order to avoid compromise of the test, 
and that interim tests of performance objectives would provide part of the basis for 
payment. The choice of tests was later to become a matter of contention because of 
probable mismatches between the content of the tests and of the instruction. Origi- 
nally, ETS had proposed that LRA witness pre- and post-test conditions, a provision 
that was replaced by one stating that Norfolk schools would certify the conditions 
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of the testing in writing. This was also to become a matter of contention at the 
completion of the program. 

Exact conformance to the specifications for construction and administration of 
the interim tests would have required a very large effort by the evaluator, because 
each student might need to be tested on a set of objectives different from those for 
any other student- To avoid this difficulty, the sampling procedure already noted in 
the amendment was devised. Another major revision occurred when the interim 
tests were all administered within a few weeks, rather than at relatively equally 
spaced intervals. This change in schedule probably came about because LRA was 
slow in supplying the tost items and the University of Virginia could not cope with 
the logistics of the more widely spaced testing once the test items had been agreed 
upon. 

Sections 4.06 and 4.07, which are added to the ETS '’raft, afforded LRA valuable 
protection against the eventualities that there would not be enough time or students 
upon which to base payment. That such protection is needed has been demonstrated 
by both the Norfolk program and other performance contracts. 

A clause that has turned out to be quite significant is the provision that on the 
completion of the program the Norfolk City Schools may buy outright the LRA 
materials, diagnostic tests, and other elements of the instructional program in 
whatever quantities they want, but that this option must be exercised by July 15, 
1971, Since that date was well before the results of the evaluation report would be 
made public, this meant that publicity would not influence their decision. 
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VI. THE PROGRAM IN OPERATION 



CHOICE OF PROGRAM AND COMPARISON SCHOOLS 

Once Norfolk had been chosen as a division for performance contracting, the 
school division administration was asked to select the Title I schools that would 
participate. They originally chose two junior high schools, Jacox and Ruffner, and 
two elementary schools, St. Hele a and Bowling Park, with high percentages of Title 
I students. Jacox and St, Helena ere to provide the treatment groups, and Ruffner 
and Bowling Park the control groups. However, the redistricting left Bowling Park 
with only the first through fourth grades and decreased the number of Title I 
students at Ruffher; therefore. Young Park wa.s chosen as the control school for St. 
Helena and Campostella for Jacox. Young Park vms among the elementary schools 
that had not been desegregated; also, Campostella was not desegregated because 
bussing students back and forth across the river was considered too difficult. 



CHOICE OF PROGRAM AND COMPARISON STUDENTS 

In order to qualify for the treatment group, a student had to score below his 
nominal grade placement on the most recently available district reading test score 

The methods of selection of students for the "control” groups vitiated the applicability of the term 
"control" to these groups, in the sense of experimental design. Therefore, these groups will be referred 
to as "comparison” groups hereafter. 






and preferably would have an IQ of at least 75, again based on the most recently 
available score. For students who had participated in the regular testing program, 
therefore, the criteria were met on the basis of scores provided by the tests listed 
in Table 5. 

As noted previously, however, almost all of the children in the fourth, filth, and 
sixth grades at St. Helena were in the treatment group from the outset of the 
program. By Marcli, in order to replace a few students who had been dropped 
because of excessive absences,^® all of the St. Helena students in the upper elemen- 
tary grades who were reading below grade level were in the program, whether or 
not their IQs weie below 75. 

At the beginning of the program, 126 students were selected for the treatment 
group at Jacox. Of these, dO had been in the remedial reading program the preceding 
year and 65 more were on the basis of the stated criteria. A student’s 

disciplinary record was not considered in any way in assigning students to the 
program^ The absence rate at Jacox almost guaranteed a high rate of turnover; in 
fact, of the original group, about 70 percent completed the progi am, 

Although a few Jacox students were designated as "reserve” (i.e., to be drawn 
on if replacements were needed) and were pre-tested, these were quickly exhausted. 
Since the University of Virginia could not handle the logistics of unpredictable 
pre-testing, only 14 percent of the late entrants received a pre-test. 

The "control” group at Campostella Junior High was selected at random from 
the seventh grade (only) by choosing every tenth seventh-grader. Because the cri- 
teria for selection were so different from those for the treatment students, this group 
is referred to as the comparison group, rather than the controZgroup. Campostella’s 
seventh grade contains 14 classes, with a total of 430 students. Thirty-eight students 
were pretested for the comparison. The comparison group at Young Park was also 
chosen at random, but from the fourth, flffh, and sixth grades. One hundred and four 
students were pre-tested for the comparison. 

Since it seemed to us that the students in the remedial reading programs at 
Young Park and Campostella might be more like the students in the treatment 
groups at St. Helena and Jacox, we gathered data on these students’ IQs and division 
reading scores. 

Table 6 displays the number of program students by grade, their mean IQs, and 
reading scores (expressed as number of grade levels above or below nominal), and 
thair role in the program. Also shown are data describing some of the students in 

Excessive absence was defined to be any absence of 10 consecutive days or more, or any absence 
of 16 days total. 
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CHARACTERISTICS OF STUDENTS PARTICIPATING IM PERFORMMICE CONTEACt'MG 
OR REMEDIAL READING PROGRAMS 
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the remedial reading programs, along with their mean IQs and mean reading score 
deviations. The table suggests that the treatment students in St. Helena and Jacox 
were, in fact, quite deficient in reading and these deficiencies increased with age. A 
number of treatment students had IQs below 75 (26 percent for the fifth grade and 
28 percent for the seventh and ninth grades). The high ratios of male.= to females, 
which obtained consistently at the junior-high level, including those chosen at ran- 
dom, may not be the result of chance alone. They may reflect a propensity of females 
in this population to leave school early, but we have no data to refute or substantiate 
this point. 

A comparison of Table 6 with Table 4 indicates that the treatment groups had 
lower IQ scores than the general student population in the treatment schools — 
considerably lower in the case of the junior-high students. It is more difficult to 
compare the scores on the division reading tests. The SAT is not at all comparable 
to the SRA because of the 3.1 g-ade equivalent "floor” on the SRA. Although the 
seventh- and nin'^’i-grade reading scores appear considerably lower for the treat- 
ment group than for the Jacox population as a whole, again the question of compara- 
bility between the SAT and the Iowa Silent makes it difficult to draw a firm conclu- 
sion. Nine students in the ninth-grade treatment group had an average score in the 
seventh grade on the Iowa Silent of —2.3 below seventh-grade level; the figure for 
the Jacox so^'enth grade as a whole was —1.6. 

Table 7 displays some two-way comparisons of differences in IQ scores for stu- 
dents participating in the performance contracting program (treatment and com- 
parison groups) and students in the regular remedial reading program The t-test 
was applied in an attempt to determine whether the several groups were signifi- 
cantly different on the basis of IQ. The results suggest that the fifth-grade students 
in the Young Park remedial reading program were better matched to the fifth- 
graders in the St. Helena treatment group than were the students in the Young Park 
comparison group, whose IQs were significantly different at the 0.10 level. Interest- 
ingly enough, the Young Park remedial and comparison groups differed in IQ at the 
0.06 level. 

Since, in order to keep the demands of data collection within manageable 
bounds, we had decided to emphasize grades five, seven, and nine in our work, and 
because of the preponderance of students in the eighth grade in the Campostella 
remedial reading program, tiiare were too few students in the seventh and ninth 
grade remedial program to make comparisons similar to those above. The compari- 
son of IQs between the seventh-grade students in the Jacox treatment group and 
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TWO-WAY COMPARISONS OF DIFFERENCES IN IQ SCORES OF STUDENTS 
PARTICIPATING IN PERFORMANCE CONTRACTING 
OR REMEDIAl. READING PROGRAMS 



Group 


Significance Level® 


5th 

Treatment 


5 th 

Comparison 


5th 

Remedial 


7 th 

Comparison 


5th Treatment 
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.10 


.50 
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5th Comparison 
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.05 


X 


7th Treatment 


X 


X 


X 


.40 



£ 

C-tests of significance. 



Campostella comparison group supports the belief that these groups matched each 
other fairly well. The lack of comparison groups at the eighth and ninth grades was 
a serious omission, however. 



TEACHER SELECTION AND TRAINING 

The project coordinator for Norfolk City Schools, Daniel A\ ent, proposed the 
names of four secondary teachers and six elementary teachers from whom LRA 
would choose one secondary teacher and one elementary teacher for the treatment 
groups. No guidelines for selection were given except that each teacher should want 
to be involved in the program and should be flexible. Stern made the final selection 
after he had interviewed each nominee. He based his choice primarily on the crite- 
rion of the nominee’s responses to "What if?" questions. Both of the Norfolk teachers 
demonstrated a high degree of competence in putting the program to work in their 
classrooms. 

Teachers attended a week’s pre-service training given by Cohen in Farmville, 
Prince'Edward County, The first day was spent in enlisting the teacheis’ cooperation 
by discussing the good points of the program. Next, they spent a day and a half 



35 , 



working through a flow chart of the system and materials. The remainder of the time 
was spent in teaching actual students. Cohen believed that some of this orientation 
could have been dispensed with; in addition, the teachers felt that the workshop 
could have been more helpful. The problem was probably one of emphasis — too 
much time was spent on conceptualizing and not enough on the details of testing, 
diagnosis, prescription, and management that the teachers had to face, Po.ssibly 
LRA had not worked out all the details at this early point in the program. 

LRA had three field supervisors for the program, indigenous to the areas in 
which they worked. Their major job was to enlist and maintain the support of the 
school principals and to insure that the program was implemented properly in the 
classroom. Initially, they worked Aill time, and for the first three weeks they tried 
to visit each classroom every day. Apparently, both teachers felt that they had 
received sufficient assistance and guidance from LRA, although one did concede that 
follow-up conferences with LRA would have been usr'ul as the program progressed. 



THE PROGrAM IN THE CLASSROOM 

Classes at both the elementary and junior-high levels contained about 26 stu- 
dents from a mixture of the grades included in the treatment. Each teacher taught 
five classes a day and had no other duties beyond the. o attendant on the program. 
They estimated, however, that they spent between one and two hours a day outside 
of class working on things related to the program, such as filling out reports, prepar- 
ing materials, and the like. During class each teacher was assisted by a full-time aide 
who helped keep materials in order and equipment working, assisted students hav- 
ing difficulties, and performed clerical duties. One of the teachers had special ses- 
sions on Saturday for students to make up missed periods or to do extra work; the 
other teacher allowed students to attend extra sessions during the day as they 
wished. The former seems to have been the more effective, as the 42 fifth-grade 
students each attended an average of 3.2 extra or make-up sessions whereas very 
few of the junioi -high students attended extra sessions. 

To participate, students at the junior-high level had to forgo their elective. At 
the elementary level students in each grade were placed in three groups, each of 
which left the regular class at a different time, making it difficult for the classroom 
teacher to plan her day. 

The original plan for the course of study was to use a set of 300 objectives 





prepared foi' nonreaders or poor readers at the junior-high level by Cohen in his 
Mobilization for Youth program in New York City. Cohen’s program included much 
material to help the ghetto child develop the listening and speaking vocabulary he 
would need before he could develop a reading and writing vocabulary. This need is 
much greater than the layman appreciates. A diagnostic test would be designed from 
the materials to be used, in order to determine at what level each student should 
be working. The levels were preprimary to third grade, third to fifth, and fifth to 
twelfth. This test was not available at the beginning of the program, so the teachers 
began work from their knowledge of each student’s grade level in reading from the 
scores on .he division reading tests. 

Toward the end of the first month, the diagnostic tests began to arrive. Of the 
original 300 objectives, 240 remained after 60 that were deemed irrelevant had been 
deleted. Objectives were grouped in three areas: word attack, comprehension, and 
study skills. The objectives in word attack followed a fairly well-defined sequence, 
but students could skip about in those for study skills and comprehension. The easier 
objectives were developed in the most detail, which is reasonable, as most students 
did not progress very far into the comprehension objectives. 

At the same time, stress was laid on reading books, and a wide variety of books 
was supplied to encou'fage students to read as much and as freely as they wished. 
Some of the books were black -oriented. The junior-high teacher felt that the free- 
readin^, book conferences, and rewards for reading books (to be described in more 
detail shortly) were very eflfective in holding the students’ interest until the end of 
the program. 

The theory behind (Dohen’s approach is that the most efficient instruction is that 
v/hich is most closely tailored to each student’s needs. In order for the teacher to 
attain this end with a class of 25 students, special materials are needed that have 
been designed so that students can work with them independently. The teacher 
cannot personally attend to each student’s needs. She also must have ways to check 
student progress, keep student folders in order, and keep materials and equipment 
in order so that students can get materials and put them away as they need them. 
Each studeiit h is a card on which the teacher has entered abbreviated designations 
of the materialb he should be working on and crosses them off as he finishes them. 
The teacher works only with students who are having trouble and also encourages 
students to teach each other. Students move about the room as necessary so that 
they will not sit idly while waiting for the teacher to get them what they need. 
Generally, they sit and work where they please. 



This approach appeared to function very effectively in Norfolk. Students did 
work individually, got their materials as they needed them, developed a sense of 
responsibility for their own work, and helped each other. In contrast to the regular 
classrooms, most of the students seemed to be working most of the time. Usually 
about a fifth to a third of the students were moving around the room while the rest 
were working. The teachers said that it took about a month to train the students 
to work in this fashion. 

As with a number of other programs that use an individualized approach, Li.A’s 
instructional techniques included tangible rewards to students for certain behaviors 
— in this instance, for finishing objectives and for self-motivated reading. Books and 
certificates were used as rewards, w.ith their effectiveness depending to a large 
ej tent on the individual student. Students received a free book when they had read 
their first five, and a free one thereafter for every ten books read, on the theory that 
if the things it is desirable for the student to value are used as rewards, he will, in 
fact, come to value them. For some students, of course, this tactic will not work and 
rewards closer to their existing value structure will have to be used at first. Students 
were uninterested in certificates for completion of objectives; this feature needed 
more wc "k, according to one teacher. Apparently, however, the free books were 
successful with a large number of the students in Norfolk. By the fifth month of the 
program, 23 books had been given away at the elementary level and 250 at the 
junior-high level. 

LRA s program was rich in materials and equipment, paralleling in variety the 
materials used to stock the remedial reading classrooms. At the elementary level the 
most frequently used of these materials were, in order of use: the Michigan Lan- 
guage Program (to teach listening skills via tape recorder), the Barnell Loft work- 
books, the Random House books, Skilpacers and cassettes, SRA kits, and the Scott- 
Foresman First Talking Alphabet. The phonics Write and See books, published by 
New Century, were supplied to each child. The students also kept the tape players 
for practicing listening skills in constant use and enjoyed them greatly. 

At the junior-high level, the Michigan Language Program, which develops lis- 
tening skills, was useful lor only a few of the students, as most of them were above 
that level. Even so, most of the students got only about halfway through the compre- 
hension objectives. The last months of the program were spent primarily in reading 
and discussing books, rather than working on specific skills. 

The elementary reading center was a regular classroom furnished with two 
teachers’ deskE, about six tables, a lonsr table with ten carrels that the principal had 
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made, two tables containing miscellaneous books for free reading, six small tables 
containing study materials, wall displays of student progress, a four-drawer file, and 
a six-foot storage cabinet. The junior-high reading center was similar, but was in a 
separate trailer (about 24 by 18 feet) outside of the main school building. The trailer 
also contained a carpet and a room air-conditiokier, which was not needed at the 
elementary center because the "ntire building was air-conditioned. 

Botli of the teachers believed that the program was not optimally designed, and 
that fewer classes with more students per class would be better. Given the proper 
facilities and equipment, the teachers estimated they could manage 35 to 40 stu- 
dents per class. However, the aide would still be needed. 



OUTSIDE INTEREST IN THE PROGRAM 

If performance contracting is truly to be a change agent, the approaches o. 
specific programs should spread to other similar student populations. For this to 
occur, terjhers, principals, and administrators not directly involved with the pro- 
gram need become aware of it and to foster its dissemination. 

There are two facets to the question of the attitude of such outsiders toward the 
prr 'ram. One has to do with the amount of effort that has been expended to promote 
interest and a favorable attitude, and the other has to do with outsiders’ direct 
observations of the effects of the program on students assigned to it. With regard to 
this first issue, LRA did not have an extensive plan for publicizing the program, 
involving parents in program activities, or other positive steps for enlisting support. 
Therefore, whatever steps were taken were largely contingent on the people in- 
volved at the level of tiie school division and below For example, at the outset ot 
the program one building principal had a faculty meeting to explain the program 
and to ask teachers to take an hour during the day to observe it; at that same time, 
in another school none of the teachers but the reading center teacher knew anything 
about the program, and in the comparison schools not even the principals knew 
anything about it, although one vice-principal had read a newspaper article describ- 
ing it. Later in the prograin, the reading center teachers had succeeded in eliciting 
considerable interest on the part of other teachers in the schools, including formal 
in-service training sessions on Saturdays, informal faculty meetings to explain the 
program, and encouragement of droj -in visito. The Program Director for Norfolk, 



Daniel Avent, had set up a schedule of regular visits for teachers (primarily reading 
teachers) throughout the division, from neighboring divisions, from Old Dominion 
University, from the State Department of Education, and so on. By this .ime, all of 
the reading teachers in Norfolk had heard of the program and almost all had visited 
one of the reading centers or were scheduled to do so. 

On the other hand, relatively little was done by the school division to stimulate 
parent interest and participation. Because parental permission is needed to take 
students out of electives in the seventh grade, a form letter was sent to each parent 
requesting this permission. After a rash of absences in the winter, notices were sent 
out to parents to try to get better attendance in the program. Some parents came 
to school in response; others never received the notices because their children would 
not take them home. One teacher had a parent conference two weeks after the start 
of the program and another in March. Aside from these episodes, most parents did 
not express interest in the program directly. An attempt to conduct a telephone 
survey of parent opinion toward the end of the program was frustrated by the small 
percentage of telephone numbers provided on the student records. 

The project director believed, and we concurred, that it was important to main- 
tain records of visitors in order to have as much information as possible that would 
cast light on the interest and involvement generated by the program. Avent devised 
a form on which the program teachers would maintain a record of visits to the 
Center;®^ this form also had to be filled out and submitted first to Avent by anyone 
planning to visit the Center. When the Centers were visited in the spring, however, 
we discovered that only one of the Center teachers was actually keeping the log. The 
other simply assumed that Avent's records would be sufficient, which, of course, 
means that the data are probably in error. Some people may have requested permis- 
sion to visit and then did not do so; others (the author included) may have visited 
without filling out a form. 

In addition, the principal of one of the treatment schools initially intended to 
maintain a log of events pertinent to the progr. im; Rand felt that this would be an 
extremely valuable source of information, Untbrtunately, the press of work and 
events forced him to abandon this endeavor. 

Several teachers who had students in the performance contracting program 
were interviewed to determine the extent of their knowledge of the program and 
their opinions of it. All had positive thin^ to say about it, although a few noted some 

Rand was asked to construct a form that would have gathered more detailed information; appar- 
ently this waa too cumbersome, as the fornri provided was not used. 
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deficiencies and dangers. In general, they believed their students had a better atti- 
tude toward reading, as evidenced by increased use of the library, greater willing- 
ness to undertake assignments that involved reading, better listening and word- 
attack skills, and eagerness to attend the reading center. On the negative side, at 
the elementary level, teachers doubted that students were leally reading for mean- 
ing and that comprehension was being underemphasized. None of the teachers 
noticed any particular change in the students’ attitudes toward school in general. 
And at least one teacher expressed unease at the thought tijat people other than 
professional teachers might assume major classroom responsibilities. 
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VII. EVALUATION 



VALIDATION 

Because the primary responsibilitj' of the Bureau of Research at the University 
of Virginia was to certify student achievement, we have chosen to term their ser- 
vices validation, rather than evaluation, which has a much broader connotation. 
Although the evaluation contract contained provisions for widening the scope of the 
work, by and large the importance of validation overshadowed other activities, with 
resultant cursory treatment in some instances. 



Achievement Test Selection 

We do not know on what basis the standardized achievement tests were selected 
by the Bureau of Research. In most instances, the levels of the tests were quite 
appropriate to the average attainment of the target population, but the content of 
the tests poorly matched the content of the instruction, particularly at the elemen- 
tary level. For e^iample, although many students at that level spent a great deal of 
time developing word-attack skills, the word-attack section of the Metropolitan 
Achievement Test (MAT), which is given orally, was omitted, probably because of 
difficulties in training test administrators and because none of the other tests used 



had a cornparable section. Presumablys a diflTerent form of each test was adminis- 
tered pre and post. To our knowledge, the tests we;re administered as shown in Table 



The most striking feature of Table 8 is the large variety of tests and levels of 
tests administered. Because the Norfolk schools have not been given the details of 
the pre-test administration, it is also not possible to know whether all students even 
had pre- and post-tests from the same publisher. The variety of tests makes it very 
difficult to compare the achievement of individual students or groups of students 
because each test measures somewhat different skills, may have been normed on a 
different student population, and is constructed differently. For example, tests have 
different numbers of questions and different numbers of choices per question. Be- 
cause the correlation of scores on one level of a test with those on another is less 
than perfect, the reliability of a "gain score” obtained in this way is in doubt. 

Achievement Test Administration 

By and large, tests were administered at least as carefully as achievement tests 
are usually administered for norming purposes, based on Rand observation of the 
prertesting. Although there were some instances of poor administration, inadequate 
facilities, and poor arrangements for getting students into the test rooms, these were 
probably no better or worse than usual for either the pre- or post-tests. Student gains 
were poorest in St. Helena, where test administration was handled with the most 
care;®® this suggests that inept test administration is not to blame for all the poor 
results. 

Based on preliminary data, and assuming that all post-tests matched the pre- 
test codes (except for changes in form) provided to the Norfolk schools in February 
or March, the number of students who took post-tests and pre-tests that matched in 
publisher and level is shown in the table below. It is clear that the reliability of gain 
scores is very low for many of the tests. In addition, the size of "valid” samples 
(where "valid” is taken to mean administration of matching testa) is much too small 
in some instances (particularly at the junior-high level) to support statistical signifi- 
cance. 

According to reports by the test administrators. 
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Table 8 

TEST ADMINISTRATIONS BY THE BUREAU OF RESEARCH, 
UNIVERSITY OF VIRGINIA 



Grade/ 

Adminis- 

tration 


Test Namea^ 


Forms 


Level 


4/both 


California Achievement 


A&B 


2 


4/both 


Metropolitan Achievement 


G 


Primary 11 


4/both 


Science Research Associates 


D 


Battery 2-4 


4/post 


Science Research Associates 


D? 

: 


Blue of Multilevel 
Battery 4-9 


5/both 


California Achievement 


A&B 


2 


5/post 


California Achievement 


B 


3 


5/both 


Metropolitan Achievement 


G 


Primary II 


5/both? 


Metropolitan Achievement 


G 


Elementary 


5/both 


Iowa Test of Basic Skills 


4 


3 (multilevel) 


6/pre 


California Achievement 


A&E 


3 


6 /both 


Metropolitan Ac hi e v emesi t 


G 


Primary 11 


6/both7 


Metropolitan Achievement 


G 


Elementary 


6/both 


Iowa Test of Basic Skills 


4 


4 


6 /post 


Iowa Test of Basic Skills 


4 


5 


7/both 


California Achievement 


A&B 


3 


7 /both 


Metropalican Achievement 


G 


Intermediate 


7 /both 


Iowa Test of Basic Skills 


4 


5 


8/both 


California Achievement 


A&B 


4 


8/both 


Metropolitan Achievement 


G 


Intermediate 


8/both 


Iowa Test of Basic Skills 


4 


6 


9 /both 


California Achievement 


A&B 


4 


9/both 


Metropolitan Achievement 


G 


Intermediate 


9/both 


Iowa Test of Basic Skills 


4 


7 



®In each case the vocabulary and comprehension sections of 
the test were administered. 





Grade 


Test 


Niirriber of Students 
Matched /Total Tested 


5 


CAT 


13/19 


5 


MAT, Primary II 


14/14 


5 


ITBS 


14/15 


7 


CAT 


20/20 


7 


MAT 


7/16 


7 


ITBS 


9/15 


9 


CAT 


8/8 


9 


MAT 


2/9 


9 


ITBS 


5/18 



Results of Achievement Testing 

The actual results of the achievement testing have not been made available to 
Norfolk. However, on the basis of the data that have been released, (again assuming 
that all pre-tests matched post-test codes except fbr changes in level or form), the 
program appears to have resulted in the average gains^* shown in the table below. 
(Only those students who we are certain took both a pre- and a post-test are in- 
cluded.) The table suggests that there are consistent differences among the tests for 
this population and instructional program. The ITBS shows the largest gains, the 
CAT next, and the MAT the sm;allest. This may be due to a very poor match between 
the MAT and the content of the instructional program, a problem that has arisen 
elsewhere for programs using Sullivan reading materials, according to Dr. Richard 
Harsch of the Educational Testing Service. 



Grade 


Test 


Number of Students/ 
Average Gain in 
Grade Equivalent 


6 


CAT 


11/0.2 


5 


MAT 


15/ -0.2 


6 


ITBS 


11/0.5 , 


7 


CAT 


11/0.4 


7 


MAT 


6/0,5 


7 


ITBS 


8/0.6 


9 


CAT 


6/0,26 


9 


MAT 


2/0,05 


9 


ITBS 


1/1.1 



Although there is little reason to believe that these figures are grossly in error, the gains on specific 
tests may be attributable to the Tact that the precise specification of the pre-tests is not known. 
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We cannot be sanguine about the apparently better results of the ITBS, how- 
ever, because many of the scores on this test were in the "chance range,” that is, 
the grade equivalent corresponded to a raw score that could have been obtained on 
the average by guessing. Nearly half of the pre-test score;- for the seventh and ninth 
grades were of this nature; this had dropped to about a fifth of the scores on the 
post-test. The MAT in the seventh grade also suffered this defect. Three out of the 
eight pre-test .scores v/ere in the chance range, as were three out of ten of the 
post-test scores.®® Although scores in the chance range appeared on the other tests 
or in other grades, the above are the most serious eAamples. These data further 
increase our disquiet as to the reliability of the data on the basis of which LBA will 
be paid. 

There were enough data on the fifth-grade students in the St. Helena treatment 
group to permit investigation of whether the test used affected the students’ gain 
scores (again, assuming our decipherment of the pre-test codes was correct). Figures 
4, 6, and 6 show scatter plots of gain scores versus IQ (Lorge-Thorndike) for the MAT, 
CAT and ITBS. The plots appear strikingly different but although all coefficients of 
correlation were negative, the correlations were ail too weak to be significant. (The 
figures were derived from a set of data slightly different from that used to compute 
the mean gains above. The overall results are essentially the same, however.) 

Table 9 displays the results of two-way comparisons of the students in the three 
test groups. The t-test was used to compare IQs and gain scores. The table shows that 
all three groups were about the same in IQ and that the gain scores were not 
significantly different except for those on the MAT and ITBS. These latter v/ere 
significantly different at the 0.001 level! Thus, the MAT and ITBS may have been 
measuring different skills, a difference that may have been accentuated by the 
instructional progrti.m. 

One problem — that of student absences, which plagues all testing programs, 
particularly for Title I students — ^worked to destroy what vestiges of reliability there 
might have been in the evaluation data, which used pre- and post-test averages for 
students absent at the time of testing and for students who entered the program late 
or left it early.®* This practice is hardly defensible, because such students may be 
among the better learners in the program (or the poorer, depending cn selection 
criteria). For example, the results for the ninth-grade students were quite different 

°° One of the test administrators was astonished at the speed with which some children completed 
the test. Some took half the time allotted; were they marking their papers at random? 

31 Very few of the late entrants (or early leavers) had pre- (or post-) tests. What tests were given were 
administered by guidance counsslors. 
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Fig. 4 — Scatter plot of gain scores on the Metropolitan 
Achievement Test versus IQ (Lorge-Thorndike) 
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Fig. 5 — Scatter plot of gain scores on the California 
Achievement Test versus IQ (Lorge-Thorndike) 
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Fig. 6 — Scatter plot of gain scores on the Iowa Test of 
Basic Skills versus IQ (Lorge-Thorndike) 
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Table 9 



two-way COI-IPARISOffS OP STUDENTS T/JCING 
THKE'K DIFFERENT STANDARDIZED TESTS 





Test -md Significance Level''^ 


Test 


MAT 


CAT 


ITBS 




A. Comparison of 


IQs 


MAT 


X 


N.S.b 


0.20 


CAT 


X 


X 


0.40 


B. Comparison of Gain Scores 


MAT 


X 


N.S. 


0.001 


CAT 


X 


X 


N.S . 



^t-test of significance. 

N.S. = not significant at any 
level. 



from those shown on p. 45 when pre- and post-test averages were used. These data 
are contrasted below: 



Test 


Number of Students/Average 
Gain in Grade Equivalent 


Students with 
Both Pre- and 
Post-test Scores 


All 

Students 


CAT 


6/0.25 


8/1.2 


MAT 


2/0.05 


11/ -0,2 


ITBS 


1/1.1 


17/1.2 



Interim Tests of Performance Objectives 



The original instigation to include interim tests of performance objectives as 
part of the basis for payment came from ETS. This was a rash decision in light of 
the facts that: 








• Few such tests have been developed (LRA constructed the test items after 
the start of the program); 

• Their reliability is unknown (a few items were ambiguous); 

• Their validity (i.e., their correlation with the tests of more general skills, 
which predict future academic attainment) is unknown; 

• Efficient techniques for administering such tests have not been worked 



All of these facts caused difficulty. When the first interim test came due in 
November, it became clear that the requirement as stated in the contract could not 
be met. Theoretically, since instruction was individualized, each student would be 
responsible for a different set (at least in part) of ten to twenty objectives. In order 
to measure within the accuracy required by the stipulation that the student demon- 
strate a mastery of 85 percent of the items relevant to a given objective, twenty test 
items would be needed for each objective. Even if all students had the same twenty 
objectives, this would require a test containing 400 items. Therefore, Woodbury, 
leader of the evaluation team, Blaschke, president of ETS, and Avent met to resolve 
the problem. 

First, they decided to reduce the requirement for passing from 85 to 80 percent 
so that only five test items would be needed per objective — that is, a student would 
need at least four out of five right answers rather than seventeen out of t\/enty to 
pass. Next, LRA was requested to provide a set of test items that corresponded to 
the program objectives. These arrived in late February or early March, at least three 
months behind schedule. Then it was decided to use only a sample of the students 
and a sample of the objectives assigned to these students to reduce the testing load. 

The Bijreau of Research hired and trained two graduate students from the 
University of Virginia Extension to administer the tests. The Bureau also obtained 
a list of students and their objectives from the reading teachers, and the testers 
constructed a matrix of students and objectives in order to group studente who had 
the most objectives in common for specific testing sessions. At most nine students 
were tested at one time; each test took no longer than an hour for each student and 
was not a speed test. Students responded by marking with a grease pencil on acetate 
sheets laid over the answer page. Tests were graded on a pass-fail basis. No answer 
key was provided by the contractor. 

The tests were administered on the schedule shown in the table below. The 
results support the impression of teachers and other observers that the students 
knew what the program was designed to teach (although lack of a true pre-test 
leaves us in doubt as to whether the program was responsible for their mastery). 



out. 




School 


Date 


Students 

Tested 


Total 

Time 

Required 

(hours) 


Average 
Number 
Objectives 
Per Student 


Percent of 
Objectives 
Mastered 


Jacox 


March 24 


34 


43 


6 


90 




Aoril 16 


16 


23 


15 


84 




May 3 


13 


9 


13 


92 




May 26 


10 


9 


4 


92 


St. Helena 


March 24 


42 


38 


6 


91 




April 1C 


29 


14 


9 


82 




Ma> 3 


17 


7 


7 


91 




May 26 


11 


9 


9 


76 



AttUucle Measure 

The Bureau of Research developed an "Affective Reading Index," to be adminis- 
tered to treatment and comparison students in an attempt to determine v/hether the 
students’ attitudes had been affected by the program. Almost all of the questions 
asked the student to compare his reading class this year with the one he had last 
year. Since many of the treatment students were not in a reading class last year and 
most of the comparison students were not even in one this yea’’ n- e wonders how 
they were expected to answer such questions. 



OTHER MEASURES 

Rand gathered data on an aspect of student behavior that could be partly 
related to the students’ attitude toward the program, i.e., attendance. The attend- 
ance of students in the treatment groups at the reading centers was compared with 
their attendance at school, in general, to determine whether the reading center was 
more (or less) attractive to them than their regular classes. The data on reading 
center attendance, kept by reading center teacliers, are considered to be quite accu- 
rate. (We have included data on makeup sessions only to the extent that they offset 
missed sessions; sessions beyond that are not included.) The data on attendance at 
school in general (taken from the central files of the division) are probably less 
accurate (i.e., the absence rate is understated) because many students may answer 





the roll in the morning and then cut school iater in the day. This is a common 
practice in inner-city schools. A school year of 180 days was assumed. 

Data on attendance of students in the remedial reading classes, taken from the 
reading teachers’ records, were similarly compared with data on their general at- 
tendance. Some aspects of the results, displayed in Table 10, are striking. First, the 
low rate of attendance in contract reading centers of Jacox ninth-graders and of all 
remedial reading students in Campostella makes one wonder hew these students 
could have learned anything from these classes. Second, absence rates rise sharply 
with age. Third, the absence rates in the remedial and contract reading programs 
are significantly different from those in school in general only for the ninth grade 
in Jacox (contract) and the seventh grade in Campostella (remedial). In these in- 
stances the difference was significant at the .05 level (Jacox) and the .001 level 
(Campostella). 

Rand also made an attempt to determine whether the skills that students were 
acquiring in the reading centers would carry over into their regular classroom work 
— that is, whether they would become more proficient at handling assignments that 



Table 10 

ABSENCE RATES FOR STUDENTS IN 'raEATtENT GROUPS 
OR IN REMEDIAL READING CLASSES 



School 


Progrtm 

Type 


Grade 


Absence 


Rate (%) 


Number of 
Students 


Reading 

Class 


School in 
Generaia 


St, Helena 


Contract 


5 


7 


7 


42 


Young Park 


Rmedlal 


5 


S 


7 


21 


Jacox 


Contract 


7 


11 


10 


47 


Jacox 


Contract 


9 


23 


15 


35 


Campostella 


Remedial 


7 


30 


23 


10 


Campostella 


Remedial 


8 


21 


19 


38 


Campostella 


Remedial 


9 


18 


16 


6 



Assmes a school ydar of 180, days. No Information on 
late-entering or early— leaving students included. 



might reasonably b iven them by the classroom teacher. At the beginning of the 
program, classroom teachers in the fifth grades at both Young Park and St. Helena 
were consulted to determine what subject matter would be taught during the year 
and what would be reasonable ways to determine whether students had mastered 
this subject matter. (Difficulties of scheduling made a similar approach at the junior- 
high level infeasible.) On this basis, three assignments — one in arithmetic, one in 
English, and one in social studie-s — were drawn up and approved by all five class- 
room teachers. These, along with the written instructions to the teachers for admin- 
istering the assignments, are attached as Appendix D. 

The assignments were administered in December Z970, and the very same as- 
signments were administered in June 1971. The student responses were gathered, 
names and other identifying items removed, and a code number assigned to each 
paper. Because only 16 students turned in both pre- and post-assignments at St. 
Helena, no elaborate analysis of the results was undertaken. Rather, a teacher from 
the Los Angeles area who had had extensive experience at the elementary level was 
asked to rank a scrambled set of all of the papers for each subject from 0 (poorest) 
to 10 (best) by placing them in eleven groups. She was allowed to use any criteria 
she wished to form the basis of the ranking and to put as many papers into each level 
as she wished. Her comments are also contained in Appendix D. 

Then each student’s initial rank was subtracted from his final rank and the 
averages were computed for each school. The results are shown in the table below. 
Statistical tests show that Young Park students scored appreciably higher in arith- 
metic at the end of the year; their gain was was significant at the 0,01 level. This 
bears out other studies of black students’ ability to cope with different subject matter 
areas. None of the other gains were significant, and there was no significant differ- 
ence between the schools. 







Average Net Change in Hank 






(Anal minus initial) 




Number of 


Social 


School 


Students 


Studies English Arithmetic 



Young Park 34 0.47 -0.06 1.38 

St. Helena 16 0.75 —0.62 0.94 




COST COMPARISON 

Information on payments to contractors for instructional services, evaluation, 
and management support are of historical interest, but they are not particularly 
hfelpful in answering two major questions; How does the performance contracting 
program compare with other reading programs in terms of their requirements for 
resources? What would be the cost of implementing the performance contracting 
program as part of the regular Norfolk curriculum? 

To answer such questions, two steps r/ust be taken. The first Is to describe the 
programs to be compared in terms of their resource requirements, as shown in 
Tables 11 and 12. The remedial reading program was chosen because it is closest in 
intent to LRA’s program. The second step is to compute the cost of each program 
using the same unit cost for items common to each. Then the programs can be 
compared on a common ground, which is the most useful for planning. 

The program and resource information in Tables 11 and 12 was gathered by 
direct observation, from the Norfolk Title I budget, and from lists of materials and 
equipment. The weakest area of the de.scription is the itemizing of materials, as 
there was no way to know which of the materials were used most heavily and which 
were not used at all. This is not a serious fault, however, as the cost of materials is 
small compared with personnel costs, which dominate the operating costs of the 
programs. 

Next, the "Comparable Replication Costs,” shown in Table 13, were computed 
by S. A, Haggart, of Rand, who provided the struct ure of the resource and cost 
analysis. The comparable replication costs (rather than historical program costs) are 
given so that the planner may judge which of two pi’ograms that have already gone 
through the development phase would be the less expensive. Therefore, comparable 
replication costs do not include the cost of research and development, which is a 
sunk cost®^ for both programs. They also do not include the cost of classrooms 
(another sunk cost) or of the program director, who is Eissumed to be funded out of 
general administrative support, even though the management he provides is essen- 
tial to the success of innovative programs. The diagnostic services at the diagnostic 
center are also essential to the success of the program and are included as a major 
supporting service. 

The operational cost/student is considerably lower for the LRA program than 
for the regular Title I remedial reading program because of the higher student/ 

A gunk cost is one which has already been incurred. It can neither be recouped nor incurred again. 




Table 11 

LRA. READING PROGRAM A^JD RESOURCE INFORMATION 



Chavaater-istAos of students ..... . Title i 



Px'ogx'om aaope 

Number of students 250 

Class time 50 minutes 

Class size 25 

Number of sections ............. 3 



Facilities 

Spate 

Students/classrooin/day 

Utilization 

Furnishings 



Staffing 

Certified teachers 
Special teachers.. 
Faraprofessionals . 
Other personnel. . . 



Regular classrooms 

125 

87% 

Air conditioning, pleasant environment; 
small, modern (partitions, file cabinets, 
storage cabinets, etc., loose tables, 
chairs) 



1 per classroom 
0 

1 per classroom 
1 program director 



Equipment 6 cassette players ($25), 6 tape recorders 

($150) , earphones ($50) 



Materials .... 



Sound filmstrip sets; cassettes; workbooks 
and mlBcellaneous supplies, books, kits 



Pre-aevvioe training 

In-service training 

Othey* support .................... 

Incentives 



1 week 
3 days 

Evaluatlun — ^$10 per child 

300 paperback books given as awards 



eb!c 



56 



'^1 






Table 12 

REMEDIAL READING PROGRAM AND RESOURCE INPORMATION 
Cha.vaatBvtst-i '^3 of students . Title I 



Pi'ogrcm scope 

Nvimber of students.... 



Class time..... 

Class size 

Numbet of sections 

Faailitisa 

Space 

Students /classroom/day 

Utilization 

Furnishings 

Staffing 

Certified teachers 

Special teachers 

Faraprofesslonals 

Other personnel 

Equipment 

Mate^i^ais ........................ 

Other support 



Inoentivea . , . . 



1000 (14 elementary centers, 60 students 
per center; 3 junior high centers, 50 
students per center 
50 minutes 
10 
6 



1/2 regular classroom size 
60 
80% 

File cabinet, loose tables, chairs 



1 per classroom 
0 
0 

1 program director 
Language Master 

Books, filmstrips, games, kits 

Diagnostic center: $35,000 C3 diagnostl 

clans, 1 aide, 1 clerk, consumables) 

None 
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Table 13 

COl-IPARABLE REPLICATION COSTS FOR PERFORMANCE CONTRACTED PROGRAMS 

(Cost in dollars) 



Remedial 



Acquisition Cost 


LRA/P..:uu. .g 


Reading 


Facilities (remodel, furnish) 


Total program cost 


10,000 


— 


(Cost/ instructional area) 


(5,000) 


— 


Equipment 


Total program cost 


5,000 


13,600 


(Cost/inscructional area) 


(2,500) 


(800) 


Materials 


Total program cost 


7,600 


17,000 


(Cost per instructional area) 


(3,800) 


(1,000) 


Pre-service Training 


4,000 


“““ 


Total acquisition cost 


26,600 


30,600 


Operational Coat 

Salaries (including fringe benefits) 


Teachers ($10,000/yr) 


20,000 


170,000 


Paraprofessionals ($4,000/yr) 


8,000 


— 


Materials 


Program-related (10%) 


760 


1,700 


Consumables (student) 


2,500 


10,000 


Equipment 


Rep lacement 


500 


1,360 


Maintenance 


500 


1,360 


In-service Training 


2,400 


— 


Other Support 


Program evaluation 


2,500 


— 


Diagnostic services, remote 




29,400 


Consultants ($100/day) 


800 




Total operational cost 


37,960 


213,820 


Operational Cost/Student 


152 


214 


Minimum Student Module^ 


125 


60 



^Smallest nwnber of students that can be Includod without 
Increasing the operational cost/sCudent . 



teacher ratio. If the LRA program can be restructured so that it is more effective 
in teaching reading, ts lower cost may make it very attractive for Norfolk, 



TURNKEY 

The Norfolk City Schools are already well on the way to in piementing the LRA 
system as a replacement for remedial reading programs in the school curriculum, 
but not under performance contract. Ten reading centers, five in elementary schools 
(including the one in St. Helena) and one each in Jacox, RufFner, Willard, Campost- 
ella, and Blair junior high schools, will be equipped to handle 125 to 150 students 
each for a total of 1250 students. The materials, diagnostic tests, and so on, needed 
to equip these centers have been bought. All reading teachers at the elementary 
level and 14 reading teachers at the junior-high level participated in a two-week 
summer workshop to learn to use the program. The reading teachers from the 
performance contracting program conduced the workshop for 42 participants in all. 
The participants received $3 an hour for their work. There will be some changes in 
the balance among various materials used in the program, but its basic structure 
will be as LRA designed it. 



VIII. CONCLUSIONS 



As usually happens in complex enterprises of this kind, it is not possible to make 
a clear-cut judgment about the value of the perfonrance contracting program. 
Certainly, everyone associated with the program was bitterly disappointed by the 
poor results of the achievement testing. Tiiis outcome might have been partly due 
to unreliablity of the gain scores — a problem that has been explored at length in 
several recent papers. We believe, however, that this factor was less important 
than the probability of a poor match between the content of the tests and the content 
of the instructional program, particularly at the elementary level. Since the elemen- 
tary students concentrated almost exclusively on word-attack skills, their reading 
comprehension skills could even have been adversely affected, as the standardized 
test results seemed to imply. The criterion-referenced tests showed that the students 
had mastered most of the objectives assigaed them (although it is unclear whether 
the program was responsible for this). 

It was also clear from classroom observation and teacher interviews that stu- 
dents were learning more than basic skills. They learned to be responsible for their 
own work and to have a more positive attitude toward reading. These changes 



““ Lee J. Cronbach and Lita Purby, "How We Should Measure 'Change' — Or Should We?," Psychologi- 
cal Bulletin, Vol. 74, No. 1, 1970, pp. 68-80; Stephen P. Klein, "The Uses and Limitations of Standardized 
Teats in Meeting the Demands for Accountability,” Evaluation Comment, Vol, 2, No. 4, Januaiy 1971; 
Robert E. Stake, "Testing Hazards in Performance Contracting," Phi Delta Kappan, June 1971, pp. 
583.588. 
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seemed to be specific to the reading program and reading per se, however; neither 
teachers nor principals felt that any improvements in attitudes toward school in 
general were attributable to the reading program. In addition, the analysis of scores 
on the classroom assignmeiits in social studies, English, and math failed to support 
the theory that improved reading ability would promote improvement in other 
academic areas- 

It will be difficult to determine whether students in the treatment groups did 
or did not improve more in reading than their peers, primarily because of the lack 
of a true control group. In a few instances, the small sample sizes resulting from the 
use of a variety of tests also make comparisons difficult. An inadequate evaluation 
plan therefore hampered the evaluation, as did the requirement to maintain test 



security. 

The need to maintain test security also kep t the evaluator from supplying the 
teachers with any interim data on student performance, and thus thwarted one of 
the major uses of evaluation — program improvement. This is especially important 
for a program undergoing development, as this one was; an interactive evaluation 
might even have caught the imbalance of program content at the elementary level 
in time to correct it. 

Perhaps the m^'or difficulty was that the evaluator faced too many and too 
varied tasks, spread out over too large a geographic area. The establishment of 
on-site evaluation teams could have mitigated this problem, and scrupulous honesty 
and objectivity could have been assured at a modest expense by having an independ- 
ent audit of the validation. 



At this point, we should discuss the difficulties of gathering data in the public 
schools. (Norfolk is no worse in this regard than any other division.) Lack of readily 
accessible, complete data forces an evaluator to spend most of his time grubbing in 
files iii principals' offices, Leacikers’ recorci hooks, aiid so oii. Even data tliat are fed 



inio the central computer are not kept in a central file, but are lost once they have 
been entered on this or that record. The result is a great waste of time that could 
be spent interviewing teachers, talking with parents, observing classes, and the like. 
Furthermore, what data are gathered are likely to be incomplete and often wrong. 

Wlien special data are to be gathered as part of an evaluation, it is false 
economy to assign the task to classroom teachers, building principals, guidance 
counselors, or the like, who are likely to have other extra duties generated by the 
program and whose primary responsibility is not to gather data. Only the most 
conscientious person will faithfully provide complete records in such circumstances. 



f 
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The evaluator would be much wiser to put someone on the scene, either periodically 
or throughout the program, whose primary duty would be to collect the data needed. 
This would mean more expense, of course, but could be well worth it. 

A nu mber of problems had to do with the fact that the program was established 
by contract. The most obvious was the difficulty in providing legal arrangements fbr 
insurance covering students in the program. This delayed the signing of the contract 
and was finally resolved for a small cost in dollars but a large cost in people’s time 
and attention, including the involvement of three law firms. A contract is very 
binding in more than the legal sense; it makes it difficult to adjust the program as 
needed. Renegotiations were required, for example, when the interim testing liad to 
be revised. In short, a contract can be a hindrance when a program is first being 
implemented, unless it is written to accommodate change. 

The program cost more than it would have if it had been administered like the 
usual compensatory programs. Additional money was needed to recoup the costs of 
program development and to cover administrative expenses, travel, and the like. On 
top of this were the ETS contract for management support (probably on the order 
of $15,000, if Texarkana is any guide) and the University of Virginia contract for 
$7,400 (Norfolk’s portion of the $74,000). Neither of these expenses would be needed 
in a regular program, of course. 

Because of the larger number of students that can be accommodated in LRA’s 
system, the comparable replication cost of the program is less than the cost of the 
regular remedial reading programs. If the problems in content balance can be 
solved, effectiveness may be sufficient to warrant abandoning the remedial programs 
as they now exist. 

The performance contract brought talent from outside the school system to bear 
on a serious educational problem, and focused attention on the program. A consider- 
able amount of time and moiiey had already gone into the developnient of the LRA 
system — money that came from publishers and others in the education industry. 
The fruits of this investment weie made available to Norfolk. Additionally, outsiders 
working under a guarantee of reward for good performance seemed to be able to 
operate more freely than could have those with established roles in the school 
environment. For a number of reasons, then, the more radical departures from 
customary practice were adopted and are being extended in Norfolk’s target schools. 

In sum, it seems likely that the performance contracting program in Norfolk in 
1970-71 will result in a considerably changed reading program in target schools, a 
program that is at least as effective as the current remedial reading program under 



62 





Title I. Two steps are now being taken that may correct the deficiencies of the 
1970-71 program; restructuring the content at the elementary level for a better 
balance between word-attack skills and reading comprehension, and the use of a 
more interactive evaluation that will contribute directly to the improvement of the 
program as it develops. 
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PROPOSAL SUBMITTED BY LEARNING 
RESEARCH ASSOCIATES 
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and the 
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and the 
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FOR 

"GDABANTEED SrUDENT ACHIEVEMENT X« READING 
THROUGH OPERATION OF ACCELERATED AOllEVEMENT CENTERS" 



August 13, 1970 
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Learning Researcfa Associates 
1501 Broadway 
New -Yorh, New York 10036 
Phone: 212-594-6649 
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I. STATEMKT OF Tui: puobu:m 

The seven school divisions of Prince Edward County, 
Lunenburg County, Mecklenburg County, Wise County, Dickenson 
County, Buchanan County and Norfolk City in cooperation vjith the 
Virginia State Department of Education (hereafter referred to as 
the Virginia districts) through its request for proposal for 
"Guaranteed Student Achievement in Reading Through Operation 6£ 
Accelerated Achievement Centers" seek to achieve a number of key 
objectives. Among the most significant of these are; 

• to maximize student achievement, as indicated 
by performance in reading. „ 

. to incraase the retention pox^er of the schools, 

• to Increase the cost-effectiveness of instruction 
in reading in a manner which can be subsequently 
adopted in the schools on a turnkey basis. 

Learning Research Associates, as the prime contractor, 
seeks to assist the Virginia districts in achievemont of these 
objectlvea through the program described In this proposal. Urban 
Ed,, Inc, will sub-contract and provide assistance and support In 
Specific aspects of the Instructional program In reading and 




communication 



A, Unders tfinclln ?. of die S j. t u'.'.Ciou of the Seven. 

School Uistrlets j.i i Vi.r;.fnla 

Though variations exist amoiig the seven participating 
school districts there are a mimber of characteristics which 
they have in common. 

With the exception of KorfolU, all the school districts 
are rural and have generally declining population expectations. 
Frince Edward County may be an exception since there appears to 
be eoBie avidanco that a population increase could occur. The 
average family income and the average adult education level in 
all of the target populations Involved in this pilot project are 
below that of the State. Children in the target population may 
be characterized as "disadvantaged" and consequently the schools 
serving them have been and will most likely continue to be eligible 
for funding under ESEA Title I. Three of the counties - Prince 
Edward, hunenburg and Mecklenburg - in attempting to provide for 
the education of all their children have experienced additional 
complications related to the problems of desegregation. 

On the average a smaller percentage of high school graduates 
go on for additional training as compared with the State and the 
Matlcn. Moat of the districts have relatively poor retention power 
with higher than average drop out rates, 

A significant proportion of students in the target population 

.m 

y-u 



B1 



o 




fall Co achteva at satisfactory levels in rcadins and the related 
language arts skills. 

In light of these conditions tlsc seven Virginia school 
districts are actively seeking an approach that offers a high 
degree of likelihood of success in the reduction or elimination 
of the school problems described. A pcvforucnce contract vtith 
IneenCivos and penalties provides this kind of oj>portunlty. Such 
& contract represents one means of attracting the talents, skills 
and resources of the private sector to the aid of education and 
then holding- them accountable Cor results in the projects they 
undertake. 

In addition the Virginia districts have imposed the require- 
ment of adaptability to the turnkey process, l.e,, provision for 
the preparation, of local personnel and conditions to facilitate 
the assumption of the prograni operation in the years succeeding 
its introduction at a high level of cost effectiveness. 

This represents a much needed concept in the field of educa- 
tion acid offers the prospect of orainatlc change In schools and 
their impact on pupils in a rolotivoly short period of tima. 

The results of this project will have slgnlfilcont impact on 
the future educational planning of the seven school districts and 
the Virginia State Department of Educatton particularly as it 
relates to (1) overcoming academic deficits, (2) increasing reten- 
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tion pov7er of the schools, (3) Incteasins the cost effectiveness of 
the instructional program, (4) teacher traiulns, qualifications 
and role, (5) selection and use of learning materials, (6) design 
of physical facilities and (7) the nature and design of learning 
experiences , 

B, History of Similar Fto^rtims 

Urban Ed., Inc., the proposed sub- con tractor for the reading co;-.- 
ponenc of this proposal for the opei-ation of n.-’pld l.oaniing Centers, 
has successfully conducted eight reading projeetK Cor underachievers 
over the past three years as indicated in Figure K following page 50, 
Five of these projects are particularly relevant to the target school 
population specified by the Virginta school districts: Yeshlvn Uni- 

versity KDEA Institutes, grades 4-9j South Brons School grades 1 end 
2, Bronx, New york; r.S, 148, New York, N- Y. grades 5-S, P.J. r. Now 
York, N, Y, grade 6 and F.S. 130, New York, N. Y, grades 3-6, Ad- 
ditionally, Miss Diane Smith, a full-tliric staff member of Learning 
Research Associates, dtrecced an intensive demonstration and teacher- 
training project in Manchester, Connecticut to increase reading skills 
of Junior and senior high school students who were retarded ttJo or 
noxe years In reading. 

Finally, Ifr, Lee D, Brovm, Director of Learning Research Aseociatias, 
Inc,, participated in two adult literacy projects that successfully 
utiliaed special intensive approaches. The first of these was the PACE 
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project In Cleveland j Ohio directed by Dt . Robert Blnswangorj currently 
at the Harvard Graduate School of Education; the second was the Adult 
Literacy Project of the Milwaukee County Department of Welfare directed 
by Mr, Anthony J. SinsUy. 



C. Proposal Design to Meet the Condltlona as 
Specified in the Request for Proposal 



This proposal is designed to moot the general and 
specific provisions of the SFP for the project entitled 
"Guaranteed Student Achievement In Reading H>rough Operation 
of Accelerated Achievement Centers." 

The project is designed to operate throughout tha 19V0- 
1971 school year for a period of 150 days with each student 
participating In reading Instruction for the equivalent of 150 
hours. The pajment for the services of the contractor will fall 
within the constrairits specified in the RFP. 

The High Intensity Learning Centers In which instruction will take 
place Is designed' to provide for a significantly larger number of students, 
than anticipated in this project. Thus, in a six hour school day 
as many as 180 students can use the center without a significant 
increase in cost because Che materials are designed to be non-con*< 



Burnable, have a life expectancy of 3 to 5 years and are geared to 
an individualized approach. There is little differchca in cost in 
providing for twenty or thirty students. The only added cost would 
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be for the additional time of a teacher who might work w'th students 
during aftcr-sch j,'l or evening hours. Therefore, operating costs 
could be significantly decreased with an increase in the number of 
students participating. 

The Tearning system proposed is designed to be adopted on 
a tumUey basis at the beginning of the second year of the contract. 
This is contingent upon the assumption that professional personnel 
recruited from t,ho Virginia districts and trained by LRA and Urbax\ 
Ed,, Inc. continue to be employed in thu KitVi Intensity Luorriing 
Centers, Learning Research Associatos is prepared to guarantee 
at least tvjo alternative levels of cost-ofi'cct.ivotwss and the 
attendant conditions tjithin one r.;onth after completion of this 
project for introduction of the learning system into countci'part 
schools at counterpart grade levels. In addition LRA is prepared 
to guarantee a cost-effectiveness level of at least 507, upon 
Incorporation and adoption by the participating school districts. 

If LRA is chosen as the successful bidder a detailed survey 
will be mads imnediately with the cooperation of the approprlnte 
Virginia district personnel to determine' the nature and extent of 
all x-eBOurces - human and material - that arc available for use In 
this project. Should any financial savings ha realised as the 
result of this Bearch they will he passed on to the school district 
or diverted to provide increased effectiveness of the project. 
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LRA will be willing to nogotiate an incentive contract after 
validation of the first year achieveiiicnt results and costs to insure 
that the learning systems can. be utilized during the second and sub- 
sequent year of the project at the most cost-effective level possible. 
LRA. Is prepared to negotiate a performance incentive contract for the 
1971-1972 school year turnUey operation.^. 

Learning Research Associates and its sub-contractor, Urban Ed,, Inc. 
have extensive successful experience working with lohool personnel 
at all levels as well as with co.".iniunlty representatives, including 
individuals an.d groups. Therefore, the contiv,ctor(s) possesses 
considerable sensitivity aiid aw'areness of the kinds of approaches th-at 
are required to assure a high degree of acecptiincc of innovation 
by vorious eons titufiieics vjith a miiiinu.ti of lulvcrse reaction, 
persoTOiel of the contractor will work closely and cooperatively 
througiiogt the period of tiie contract with scliool persomol aiid 
other groups and individuals to assure a sicooth transition in the 
Installation and acceptaneo of leairnlng centers. It is our rccoiii- 
Biendatlou that to the extent po-sslble tlio Virsinla districts, within 
the existing time constraints,- plr.n on niCoLiugs with the public. In- 
cluding parent groups, to help them understand the niBrits of and 
gain support for this unique approach, LRA would wclconia the op- 
portunity to participate and contributo to these inoctings, 

LRA is willing to conduct limited prograni operations in the late 




o 



t. 
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aftcrnooii or early evenings at agreed upon sites. These opetatiois 
would be contingent upon the need for these services among the target 
or expanded population and would be reimbursable to the contractor 
on the sate oasis as in the standard compensation plan. 

In addition, LHA and "JEI, hereafter referred to as the Contractors, 
agree to the follov;ing general provisions of the IJFP: 



. all testiitg (exclusive of that intended for diagnostic 
purposes) to be conducted or supervised by each partic- 
ipating school division or by an independent evaluator 
engaged for this project. LRA has had estensiv'e experience 
in developing and administering perforrMoce tests, and in 
having clients administer perfomur.ice tests developed by Ltt\, 
Accordingly, so long as the directions for adiiiinlstrntion 
are adhered to and so certified by the project manager or 
his designee, LRA is williiig to have all lupcrvislon of 
testing used as the basis of payment under the JurisdioLion 
of the local school district. 

a report actual start-up and opci-atiug costs to 
each SupsrlnUcndent in accordance witTi the 
stipulated forms and procedures. 

. conduct operatlcnal programs within the constraints 
of, and in accordance with, the intent and eonultlons 
of the evaluation design. 
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4 tailor tha learning syeccn to meet individual needs 
o£ the target populations. 

• utilize the capabilities and resources of eacli of 
the seven districts. 

. accept the fixed time and cost constraints imposed 
by the partlcipatine school divisions and the per- 
formance criterion of luaxiinlzing student learning as 
the general basis of contractor payiiient, LRA and Urban 
Ed., Inc. personnel have a long history of meeting time 
end budget constraints while achieving specified per- 
formance objectives. Since the fall of 1969, LRA has 
guaranteed to specific school systems the achievement 
of performance objactives in connection v;ith the sale 
of its learning mater ials, particularly in the area of 
reading, 

• being willing to work with all seven districts, 

• being willing to purchase a perfortimnGe bond equal to 
the projected risk of the project as deceriniiiud by the 
participating districts. 



74 








D . Brief Ovcrvi.c;; o £ Lhe Pro pose J Ap i '_£ o^lSli 



The app!.‘ 0 £icli described is appropriate for the grade levels 
from 1 tlirough 9 Involved in this project. High Intensity 
L.eami'.'.s Centers will be established at each of the eighteen schools 
among the Sevan school districts. Approximately twenty-five students 
will bo scheduled into these centers on r dally basis for a period 
of one hour. At the outset of the project each student will ts 
diagiio'i'id to dstermina his particular needs and behavioral compet- 
encies. Appropriate instruments have been developed and/or selected 
for this purpc.'e. Subsequently each student has learning activities 
and materials prescribed for his involvement. These have bean 
selected to provide maximum responsivensss to the student's need Iv. 
ter.-is of skill, content, level and rate of looming. The materials 
are for the most part self directing and self correcting. Most are 
carefully sequenced so that students can move on independently, once 
diagnosis and prescription have been completed, with a miniinum of 
teacher direction. The teacher subsequently focuses on continuing 
diagnosis and prescription and assists students who are experiencing 
difficulty in progressing. 

The concept underlying this approach is derived from careful 
rBSaarch and subsequent application in the classroom 4.nd is based on 
maximizing the time spent by Btudonts in active iparticipation In 
learning, i,e,, ir. prescribed lEarning activitiesi. By this method 
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tile percentage o£ clock tine spent in prescribed learning activities 
Is increased Crou the avc . aga classroom rate of 35% to a lligii In- 
tensity Learning Center rate of 80 to 85%. 

Motivation for continuing attendance in tlic HILC and involve- 
ment in the learning auLivitios is enhanced by providing Ini.uadii'. to 
and direct feedback to tiie student on his responses so that he 
experiences reinforceiaont or "pay off!'. I'hc relnforcc.iiitnt sys teiu 
la translated into .charts, graphs' or the opportunity to spend more 
time in the center or in activities of tlie students’ choice . 
Additionally, the centers are doKigicd to be attractive and inviting 
in appearance and highly functional fv.u learning to take place, 

; Centers vjill have ai-ea carpets, be air conditioned, and have some 

i comfortable living room type furniture, 0ns of the cither unique 

i 

i features will be bins containing books suitable for a wide range of 

! interests and reading levels, 

i 

j The "Bystera" of high intensity learning is designed so that 

i after ttie first year of operation It can bo successfully "turnkeyed" 

: Into the regular pattern of the Virginia aehool.s at a low coat which 

is well below that of the initial contract costs. 

The total approach described is geared to the attainment of 

1 Specified periormance objectives and growth on s Uandardissed achieve- 

i 

I ment tests, Bacause of the wide range of entry ’level skills possessad 

. 

j by the target population it is difficult to offer specific guarantees 
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in terms of performance objectives before assessl.ns t\\a specific 
populations, Hov.'Cvor, wc can describe expected acliicveieent levels a. 
reflected in standardized test scores based on our previous experience , 
For tlie target population the mean aebievetuont gain t.'ill bp a growth 
of one year during the period of one year of instruction. A bank of 
hierarchically sequenced performance objectives in rending V7ill be 
available from which will be selected tbo-ce that are eppropriato for 
participating students depending upon their entry level skills. Tae 
agreed upon objectives v/ill be determined as a result of a 
review with each project rr.anager and/or tiis desi.gnntcd rep- 
resentatxve. 

Additional details a>-e presented in the section on the 
Technical Approach with further supportive mauerlnls in the 
Appendix. 



E. The Rationale of Che Conaortiua 



The consortluni undertahing this project consists of tha 
follox^ing groups with the responnlbili ties Indicated; 

Learning Resaarth Associates is the prime contractor with 
overall responsibility for project inaiiagemenc and administration. 

Urban Ed., Inc. is the sub-contractor xjJ.th direct respoiieibility 
for the teacher training and supervision of the high intensity 
learning centers for reading. 



-:r~- 
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At least tvjs Isiportant factors led to the formation of the 
present consortltmu 

1, A high degree of congruence of philosophy end practice 
exists bet^'ieen the consortium members. Ihe major eloments of 
agreement include recognition of the need for 

• individualization of instruction 

• integration of achievement motivation as part of the 
academic training of the student In school 

• application of project managemont techniques and cost- 
effectiveness analysis 

• a "iystcmatic" approach to the instructional process 

• significant modification in school practices to result in 
an improved educational experience for all students 



2, The availability of highly talented and experieneed parsonnel 
in the areas required for the success of the Virginia project. 

Supporting docuilT.viCs in the section on Corporate RncUground 
and Experience, including paisonriil. resumes, will attest to these 
factors , 



II. TECIINICAT. APrnOA CU 

The planned approacli for the project is descrlhocl in nine 
BCparnUe though intcrrola ted sections (A) Background and Rationale 
for High Intensity Learning Centers, (B) Diagnosis lii the High 
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Intensity Learning Centers, (C) Ltarning Activities in the Centers, 
(D) I'crsonncl-Sclcction, Stalling and Training, (E) Opernticu of 
the Centers, (F) Objectives, Guarantees and Cost Iinplications, 

(G) Facilities, (H) Learning tiaterials, (I) Results of EKperienue 
With High Intensity Learning Centers. 

The proposed approach is based upon several fondamantal 
assumptions. These include; 



. problems of motivation should be treated as an 

Integral past of an individual student's involvement 
in an academic discipline such as reading 
, Individualized Instruction is an effective means 
of tnaxluiizing selected groiJth opportunities for 
the individual student 

. performance objectives aitd criterion referenced test 
items are essential Co proper currlculmn planning and 
evaluation 

. a non-labor intensive approach with emphasis on 
approprlaCe materials, technology and systems and 
a changing role for the teacher is essential to in- 
creasing the effectivoness of certain kinds of learn- 



, performance contracting with the turnkey componont 
can result In dramatic changes in school practices 



Ing 
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vithln relatively sliort periods of Lii.ic 

The overall aim of t1>e propoiicd approacli is to effectively 
and afficiontly meet the objectives described in the request foV 
proposal. 

A. Bacharound and Rationale for High Intensity beaming 
Canters 

If the retarded reader is nou^u'.iito, urban or rural and 
poor, bis retardation is considered a coneo'.nltant of being dis- 
advantaged. Being "disadvantaged" weans many things a.nong which 
ares absence of a father in the liowa, spor.king vsitli a "different" 
dialect, having a deficient conceptual vocabulary, having a "non- 
verbal" eonununications style, suffering from lot? sclf-estecera, aiid 
living In an environment that does not provide stimulation for 
learning experiences. 

If the retarded reader 1* white, suburban and middle class, 
his retardation is often 1&. aled "dyslexia," or is explained as the 
result of porceptuol dysfunction. 

Black or white, the retarded reader is analyzed, prodded, 
observed, tested, exposed to instruction, re-exposed to "remedtal" 
Instruction, "diagnosed," written up and, If he is one in 100,000, 
roferred to a university clinic. 

Psychiatrists, psychologist, learning disability experts, 
pediatricians, neurologists, guidance counselors, reading speciallsta 
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and principals stipply us with the following dingnostlc labels to 
explain the retarded reader; 



Disadvantaged' 

Dyslexia 

Emotionally disturbed 
Mixed dominant 
Lazy 

Hyperkinetic 
Lack of impulse control 
Diotractibility 
Minimal brain dysfunction 
Miniir.nlly nEuro'oglcally 
impaired 

Xhe list of etiologies 



Perceptually handicapped 
or Impalvcid 
Learning disability 
Cross -'omiTiant 
Hyperactive 
Hypoactlve 
Passive agressive 
He's not ready 
Delayed meturation 



and labels may in fact describe 



conditions that exist in Individuals or groups who are retarded^ 
readers, but the accuracy of rhe labels and etiological conditions 
is Irrelevant to Che treatment of most eases of reading retardation. 



1. Research Indicates that most reading retardation stems 
from a psychoeducational disease we have labeled 
dyspedagogia — poor teaching. 

2, Even in cases of neurological Impairinent or perceptual 
dysfunction, the etiology of a behavioral condition 
that is not within the realm of madical pathology is 
usually Irrelevant to its treatment. 



In addition to documenting this point of view, this 
description presents tlie specific techniques for remadiatJjig 
dyspedagogia using High Intensity Learning Renters. 
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Ktlology of the readitig retardation (dyspedagogia) is irrelevant 
to Its treatment. 

Etiology Is crucial to behavioral research that is meant to 
pay off soaie day in preventative programs. To prevent future reading 
retardation, we must isolate the etiological variables that cause cnat 
condition, lo solve the reading problems that exist currently in 
children-, etiology is Irrelevant. 

Two other points e»ust be clarifiBO before considering the data 
in support of the irieievance of etiology. First, the specific 
conditions described by diagnostic labels and etiologies are often 
accurate. Tlic Irrelevance of etiology docs not deny the existence 
of psychosocial, paychophysicnl , psychodyna.ini c , psycholinguistic, 
economic and ecological vAriahlcs implied in diagnostic labels 
and otiologLcal descriptions. Second, ineuy of these verlables 
arc Important to consider and to treat for reasons other thair 
literacy. Hie point is, that in m:ys_t tre-a tir.enbs read ing r e. tar da - 
tlon, thes e v arinblv is arc tr r e 1 c va n t . 

According to Vskvy Austin's study,* American reading Instruction 
in general is weak. More drer,’'.tic, parhaps, than Austin's observations 
were tile classroom achlEVement level patterns that emerged from the 
Cooparatlva Readr'ng Research projects conducted in the mid 1960'a 
by the U, S, Office of Education. One of the best of these studies, 
*Mary Aostln et al.. The First R . New York, Macmillan, 1963. 
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Albert Harris'** CRAFT projeetj demonstrates the exlstonca o£ 
dyspcdagogia , 

Harris found what most other Cooperative Reading Research 
studies found and tiiat most tasearehers predicted he would find; 

Wlien wc compare various published beginning reading programs watehing 
the classrooms both with each program and across each program, the 
difference between one classroom's achievement and another's usings 
the same pr ograiu is greater tlian ciaasroom achievement levels across 
programs. Most researchers Interpreted this as; "The tender vari- 
able is more potent than the 'method' variable." This, of course. 

Is an Over-ainipli£ication, The published material is a program, not 
a method, hTmt eacli teacher does with a published program Is "the 
method." In other words, the researcli does indicate that iricthodoloa.-.- 
makes a difference in achievement, llie question that needed to be 
asked was not; Which besinning reading program gets better r-esults? 
Instead, the question should have been; What ie it that moro 
eucCBSsfuI teachers do that less ,'uccessful teacViors do not do? 

The rescarohable problem should have been: Wliat is the pcda aogv 
In the more successful classrooms? 

As an eKpcrlenccd Tescarchor, Harris anticipated the teacher 
variable or is defined rs pedagogy* lie bullu iuCO Llio GRAFT 

**Alb*rt Harris et al., A Continuation of the Craft Pro.lect ; 
Compaiflng Approaches with DlsadvantaRed Urban Negro Children 
in Prinary Grades . U.S.O.E, #6-10-063, Division of Teacher 
Education, City Univeralty of New York, 1968. 
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project attempts to isolat-7, variables that could identify more , 
successful teachers. One earlier study tnditated tvjo variables which 
appear to be most significapt; 

1. More successful teachers (as dcfli'iod by higher 
reading achievement levels of pupils) tended to 
differentiate (individuali?!e ) reading ins true tlo n 
i>;ore than less successful teachers. 

2. Classes with higher reading achieveiiient tended 
to spend taore time in reading ins true t lon than 
classes with loxrer reading achievemsnt . 

Call these variabl» dif ferehtla ted Instruction and tls^8 . The problem 
with most classroom observation scUejnss is that they concentrate on 
the wrong phenomenon -- the teacher. Whan they look at the pupil, 
they do so either in Interaction with the teacher or in an unsystematie, 
"clinical" obBervatlon. Tannenbaum and Cohen, cn the other hfnd, 
daveloped a different classroom analj'sia scheme. Instead of observing 
the teachar, they analyzed pupil behaviors in reading classes. They 
analyzed systematically what reading skills and subskills each child 
was learning, at what level he was oyierating, what channel of 
COnEnunicatlons input ho was using, through vdiat copi.i«'nir.ntion3 
channel he was respcf.idtng, the types of medio, dclivurliig the 
stimuli, the learning strategy utilized by the. lerrner and the 
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pupil groupin;; oriployotl. This U'.cliui<jUO allo./cd tha cbEtivcr to 
analyze a dess snd to dcflria net only uhat the clnes V 7 as doing on 
all eight dirianstons ] ted in the Taxoiion.iy of Ins true t.ioini'.l I'reat- 
ments (Figure A), This observation schsire revealed chat r,iore 
succeaoful classes h?/! a higher "participatlon-in- lcarning ratio" 

<P ratio) than less cuceessful classes. This study is imrorcane 
for two rcssonsj First, it focused on the learner r.nther than on 
the teacher, demonstrating that learner porf onnmice , not teacher 
performance, is the more accurate menoure of classroom efficieitcy. 
Second, it defined what diCforcnti atlon end more time meant in other 
studies that had similar findings. 

The TannenbauiU”CoIien technique indiceled chv individual iziiig 
instruction and increasing time caused rending achievement to 
increase. The factors appear to be offlcie ncs" and in tens i, tv. Given 
a period of time in which pupils are programjiiad to read, the mota 
successful classes are ones in which 85 to 90% of the clooh time l.s 
spent by the pupils workiiig in prcserlbod learning- to-read activities. 
The best vay to achieve this intensity is to adjust what a child 
learns Co each individual's needs and to allow each child to laarn 
that skill or content at his level and at his own rate . This Is 
called "indlvlduclialng" or "differentiating" instruction. The degree 
to which the teacher does not differentiate can be partially com- 
pensated for by Increasing the amount of clock hours devoted to ' 
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TAXONOMi’ OF IfTSlT.UCTIOXAL TnEAl'MJiri-S 
(Developed for Project ^OEG-l-6-0625J!{!”20J2) 
by Abraham J. Tatir.onbavm and S. Alsu Cohon 
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Study Skills 
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3. References & Texts 
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IV. COMffliNIGATIOK INPUT 

1. Auditory 

2. Visual 

3. Kinesthetic 

4. Auditory /Visual 

5. Visual/Kinesthetio 

6. Auditory/Kinesthetic 

7. Visual/Audtt/Kinesth, 



V. COi'EiUKI CATION OUTPUT 

0. . Kon-observablc' 

1. oral 

2. Motoric 

3. Oral -Motoric 
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VI. 


INSTRUCTIONAL FEDIA 


VII 


. INSTRUCTIONAL STRATEGY 


1. 


Visual-Projector 


1. 


Play - Chanca 
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2. 


Auditory-Recorder 


2. 


Play^Compa tition 
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3. 


Skill Drill Text 


3. 


Play-PuzBlo 
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4. 


Gaines -Role Playing 


4. 


Te s t-Rc sponse 
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5. 


Booki 


5. 


Explox'atlon 


Y 






6. 


Progrananad Ur ,'iponso 








7. 


Creative Proij'lca 










Solviug r . ^ 
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Vm, INSTAUCTIOXAL tUOUP I 

1. Teachev/Larg'j Croup 

2. Tencl'cr/Snali, Grc.up 

3i Tehr/Lisdiv. Sti-d/'i'iif-oi; 

4. Student/S'.unll Greup 

5. Studcnt/I.ai'r.c Creep 

6. SUurtcnC/Stu''ent ICiiiii 

7. Indlv, Self Ii'.sLriic , 
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readir .3 tii3trucui?n. In oUlior v;ordf, throe hours cir.lly ol inen'J.cient 
reading instructioai way yield a lifflo bcfter resulL than ono hour 
daily oE inefficient reading ins true Lion , lior.L trndirionnl foachcr- 
directed clcssroons operate, according to this research, at 30 to 
45% P ratios. By increasing ti.ne in tltcsc incf f icient classrooms, 
achievement increases slightly. But ^ we increase both c.f fi el eney 
and time , achiover.ient Jitjus. ’flint is precisely what happens in 
High Intensity learning Centers, 

Using the Tannenbaum-Cohen observe tion scheme results In 
the conclusion that most classrooms operate jit shockingly low 
efficiency. Using the eight taxonomy dimensions (Figure A), this 
ohservation scheme shows little real instruction occurring In most 
classrooms. "Real instruction" is systematic, sequential managemont 
of prescribed stimuli and contingencies. Most classroom instruction 
is simply eKposure. The assumption, for example, that all children 
are certainly taught letters and words does not stand up under eere- 
ful behavioral analysrs. The fact is that most children certainly 
do not receive intensive Instructloii in letters and words. They are 
merely exposed to letters and trords, and a pedagogical universe 
exists between simple exposure and systematic contingency management 
which is the key to learning. 

* Dyspedagogla is therefore, what most children get in school. 

And for most children, dyspedagogla Is good enough, for they read 
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In spite of it. Some childretij hoijover, do have uilique cowhinauions 
of negative psychosocial, psychodynamic, psychophysical, psycholin- 
gulstic variables chat requite sowathius more than dyspedagogia . 

Their problems are not in these unique combinactons, but in the 
dyspedagosia. For the Karris CRiVFT study, amongst others, shows us 
that in the presence of inte-nsivc negative etiological factors, 

a slight increase in the intensity of instruction climlnntos a 
tneasurable amount of the effect of these etJ.ological varlnhlos at 
the beginning reading stage. To assume that just because a child 
has been exposed to a course called "remedial reading" or "special 
help," he has received Intensive instruction, is an error. 

Karris* populations were disadvantaged urban children vrho 
manifest what we call in middle class children the "learning dis- 
order" syndrome. The "disadvantaged" Eyndrcme is tl learning dis- 
ability syndrome at its most intensive. In poor children we often 
call it "deprivation." In tlie CRAFT research, teachers wore closely 
trained and supervised, resulting not In optimal pedagogy, but in 
slightly better-than-averoge pedagogy. In third grade, these 
disadvantaged children in the CRAFT study were reading on grade level 

a rase phenomenon in these particular ghetto schools. The crucial 

factor was pedagogy. Stated In reverse, dyspedagogia ("low intensity" 

pedagogy) causes most reading retardation. Stated positively in 

Spite of Intensive negative psychosocial, piychodynamlc , psycho physical. 
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psycholinguis tie factors, a little better pedagogy ("high intensity" 
pedagogy) makes a big difference in achievement. 

B. Diagnosis in the High Intensity beaming 
Centers 

Students will report dally to High Intensity Learning 
Centers for v/oiii In reading and ccotiwni cat ions . 

The first step in intensive instruction, illustrated in 
the ar t of cottmunlcations, is pj.'cscrii>ti\'o diagnosis-dlstinsuieh- 
ahle from classical diagnosis by th.e former's utility. Classical 
diagnosis assigns numbers (third grade level) or etiological labels 
(developmental- dyslexia, or primary reading disorder with passive 
aggressive tendencies) to a child's behavior. Obviously, the 
classical diagnosis is useless to the child and to the clinician 
who must find a way to teach the child to read. 

The prescriptive diagnosis may also record a child's specif ie 
deficiencies by using numbers, by standard scores, or by percent 
eorreet. But the prescriptive diagnosis lies five characteristics 
that differentiate it from the classical diagnosis;. 

1, It defines the specific reading behavior ntaasurefi, 
usually hy the nature or the test used. 

2, It describes the behaviors operationally, usually 
by the nature of the test Item, ("Select one of 
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four alternatf.vo titles that expressed the paragr-aph ' r. 
main idea.") 

3. It defines the conditions of behavior on such 
duienslons as; a timed test, In a classrooni, etc. 

4. It dnfinnp, the criterion of success in such 
terms as "grade .level acliievemciu" or "percent 
correct." fhis is called eypc atc'.'.cv lc;.',cl o'ad 
and is deternined by the teacher, who considers 
the entering level of the child, hie gcvicral 
ability level, his degree of retardation and the 
level of the materials available. Hie expectancy 
level is set at e level higher than the child is 
preseirtly operating, but low enough for him to 
reach In a relatively short time , 

5. It ausuars tlie question, SO VriUlT? of its findings. 

The teacher is not allowed to record a qualitative 
or quantitative designation to any teat behavior 
unless he also records a "best guas?" SO VIIAT to 
indicate what strategy, grouping, medium or level 
(sec Figure A) will remediate the deficiency. The 
teacher usually records In the SO VnU\,T column the .. 
name of a piece of learning raaterlal to help remediate 
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the deficleacy. 




Figure B Is an cxar.iplc of a HIGH INT©;SITV LEARKiSG CEJTER SO 
WHAT Diagnosis for a retarded reading seventh grader. It cover lov;cr 
level reading skills. Another form is used for higher level or, con- 
prehension skills. That form is shotm as Figure C. The behaviors one 
chooses to list on a SO VniAT Diagnosis form "''e detor'.^Vn.-d by the ■'■’.turc 
of the pupil populations, the available teaching resources and tlio biases 
and talents of the teacher. 

The first four characteristics of the SO t’lL\T Diagnosis 
arc defined by the diagnostic tests used. The tests are parts of 
published standardized batteries, teachei'-madij checklists or 
reading samples. Using the test as the lustructionnl objective 
forces the teacher to define the behavioral outcomes and requites 
bin to admit to himself what he conceives the reading act to be. It 
prevents teachers frotn using the stereoLypud cop-outi "I 'don't 
believe In standardized reading tests. Reading Is much more than 
what the Metropolitan Achievement Test taps. 1 teach for those other 
things that tests don't measure." 

“Iha SO WHAT Diagnosis does not advocate teaching to the test. 
Instead, It demands that teachers teach for the types of behaviorB 
defined by a criterion test, Scadlng Is always "more than the test 
jamures." But whatever more one eares to teach roust, by the rules 
of the SO WHAT Diagnosis, be defined in a criterion test. For example. 
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FIGUliE E 

RAMpr.K Or SO DTA-CSiOSlS FCH 

EASIC; I;lAUJKO SIU.T.I.S >• UShT) l.'l 
Tliii) llj.vii i A Aivl't J.fiO C..t'(±]i,v 



Hame JohTi .Smith Ago 13 



Grade 7 



Date of Tests 9/&g 



Grade l.evel Fxpoctancy 





Behaviors 


Score 


so V.' 11 A.T 


1. 


Visual DiscrlTQ of Letters 


OK 


Kr.ve him \.’ 0 fk in spare Uma 
on letter form board for b, p, 

d. Ci. U, V, W, X 


2. 


Alphabet Knowledco 


OK 




3. 


Visual Memory for V.'otds 


poor 


'i’achlstoscopa Tr.alning -SDL Spellir. 
Scoll-inr; 2. 3. 4 FnychoLechnitt li'- 


4. 


Word-Recognition (Sight) 


poor 


Seme as 3; also J-?ord_ .Hank systew, 
Rolch list and Chandler Basal a 


5. 


Word Analysis 


poor 


Visual Trachins/Michir.Erj L.P. ■■ p 


f>. 


Concepntal Vocabularv 


OK 


Ko work needed now. 


7, 


Auditory Discrlm Sounds in 

vds. 


weak 


Ncdial t end we*^k, Mlcliigpin L.P- 
Aud. Dx5!Crijn, 


8. 


Phonic Spellltig 


poor 


Do 3-5 & 10-14 first. Tlicn use 
10- dictations da?Mv. 


9. 


Initial Consonants 


p.ood 


No \7ork ne.Gdcd, _ 


10. 


Final Consonants 


weak 


/ Work on 3-5, 7 first. Then pro- 

\ rr«im additional Michigan L, P. 


11. 


Initial Blends 


weak 


1 Back up with ERL and worUslise i s.-.. 


12. 


Final Blends 


poor 


? Inters »isrsc with word and 


13. 


Short Vowel Sounds 


poor 


,t card gaaa.*. If this dottn'i: woik. 


14. 


Visufi’.l Discrim* of Vov.’sls 


poor 


) U5C Foaech To print gaO'-to* O"? * 


15. 


Auditory SvHabl ficetlc"' 


eood 


No t.'ork paeded , 


16. 


Visual Svllabif ication 


poor 


Ig:iPro for no*:?. 


17. 


Structural Analysis 


poor 


After 3-5, 7 use ’ 'acii-J.l j.f.i , 


18. 


Cosiffion Confusions 


weak 


Scott Foresiaan, progrft: .“.-.Td 








Spoiling end other wo"k books_, 


20. 


Silent Reading Comprehen- 
sion 


good 


(J% code basf'oit akills.) 


21. 

22. 


l.isteniiiR Compreliunoion 


v<cdc 


lin.rnell Loft l*'oJ low T)i roc 1' f or..' 




ttons 


poor 


A & B 


23. 


FoHov;ln.' Oral lilTsctions 
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so Wli\T DIAGNOSTIC 
REPOUT FDU IlIGlI]''a 
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VOCAKULiXRY 






General 

Science 
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Social Studies 


FOLLOW DIUECTIONS 






Definitions 
Slnple Choice 
ilachomatics 
Map 


REFERENCE SKILLS 






Farts of a book 

Newspaper 

Dictionary 

Index 

Graphs 
Classifying 
Reference books 
^ps 

Organizing 
Recall of Secuence 


COMPREHENSION 






Recall of Details 
Recall of Main Ideas 
Inference 

♦Self-Initiated ReadinEt 
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teachers who ciaira to teach children "a love of booka" have been forced 
to define what they mean operationally, Hov? does one knov; ttiat 
Johnny loves to read books? In Figure C this ii.s trust ional objective 

is defined by a behavior listed as Self-Initiat e d headin g,. This is 

. \ ' 

simply an operational definition and is measured by a teacher check 
list. 

Whatever tests one uses, they should help the teacher pinpoint 
the Specific behaviors tliat must be learned and hot.' the learner is 
most likely to learn those behaviors. Many tests, including 
informal’ toachcr-mado instrunicnts , can isolate the behavioral 
deficits. But practically no test reports hovr a pupil learns. 

The best analysis of ho\.' the child may lenrn is a calculated guess 
based on how the child tends to parfoivi on a tust. That guess Is 
recorded under SO WrIAT (Figf’ B) , Wien treitmenC begliis, the 
teacher may need to correct his guess. To do so, the teacher "trius 
any combination of the eight variables listed on the Taiionomy 
(rtgure A) to engineer the types of responses defined by the cricerton 
teste. 

C, Learning Activities in I jio Cen tfra 

Intensified instruction is a system by i.-hlch a learner 
responds to prescribed stimuli 85 to 95% of the aBsJgued time. 

The prescription insures that the individual works on the speetfio 
bIeIII or iriforma tion he needs according to the SO WIAT Dlagnosia. 




Ihe indivitlual’s schedule Is simply parts of tho SO hllAT column 
(Figure fl) fitted a time schedule v.'it'n one time slot devoted 

to stjralglnt reading comprehension regardless of hoi? deficient the 
reader is in basic code busting. Thus, the throe-hour schedule 
for the seventh grader whose SO MIAT Diagnosis is shown os Figure B, 
appears as Figure D, In most cases, a child's schedule is con- 
veniently split into three areas; Word Study, Comprehension, Word 
Study Skills,* 

Examples of the materials and physical sotting for this typo 
of prescriptive High Intensity teaming Center are described later 
in this section. By structuring the learning environment with a 
large variety of specially designed materials that do not require 
constant teacher control of dclivci’y of stimuli, the teacher can meet 
the content need, level and rate of learning of each pupil v?lth a 
30 to 1 student-teacher ratio. 

A simple model is used to determine vfiich materials to use, 
how to structure the pliyslcal envirorjiian t and which stratugics to 
employ. To intensify instruction the content', level , and rate of 
learning must be adjusted to each individual. The materials, 
s tratogy and resources mus t , , . , 

*In Figure D, Tonsy le so deficient lu basic "code bunting" skillr, 
that most of his activities are eoacciitratod in Word Study. 
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FIGURE D 

CLASS SCHEDULE FOR RETARDED READER SHOT^N IN FIGURE B 



1 Word Study 1 


9 to 


Visual 


9j1S 


Tracking 


9iI5 


Taoh Training 


to 


EDL and 


9:30 


Psyclioteclinlos 


9; 35 


Michigan L. V, 


to 


Words 


X0:no 


Books 3 - A 



Cosiorchension 


10:00 


Read for Comore- 


to 


henslon In Reading 


10: 30 


Attainment (Red) 
and in Stag Up 
Cooks . * 


10:30 


EDL Lis ten Ing 


to 

11:00 


Skills Tapes 



Word Study I 




11:00 

to 

11:20 


Auditory Dlgcrin!, f 

Michigan L, P, 

Lis tening 


11:20 

to 

12:00 


Word Bank, Word 
Ga?^es to ore-tecch 
vcc-nbttlar^/ in ' 

Giandlcr irenders | 

i 



Tlies© activities may be scheduled over a three day period for one hour 
each day« 

* Regardless of how deficient a pupil is in basic T^rd analysis* a high 
interest and low vocabulary book Or kit is prOgrarRn’ed into his schedule 
to give hljn a feeling of what reading is really is. 




Personalize, 




-„for each Individual. 



'Rate 



Applied to tills model arc a mm.'jov classical Icarnifs 
principles, four of vhicli are especially impoi tant: 

8 • cl 'p pyccf al i-ir he Vi-iyjors c'.'l rvr li'i e v.’hc t 

and when t o tench hi-’i. Pi-escntly, woct schools and clinics 
pre determine viliat a child vjill learn. For cnau'plc, most first 
gredes have a set of basal readers that lavcely dcteniiinc the 
content, rate end level of instructiori, Ih.* child either fits 
the progrard at a prescribed rate and level or ho flunks. Soma 
tcac-hcrs r.ianage to divide the learners into three sub'-groups and 
write this off as "adjusting instruction to individual needs." 
This, of course. Is an absurdity, for it iiripllcs only three 
possible comb inatlons of rates, levels and content needs. Such 
Bbaurdlties contribute to dyspsdagogia. 

b. Motivation inr.naitomant is the major dotom'inant of 
learning . Ironic as it inay seetr., one of the Tnost inrldious for'cct 
to undermine learning in the claseroom is the teacher training 
InStituClou and the school administration that requlx‘o the trodl- 
tlonal lesson plan. The lesson plan is the "fifth column" of the 
school. It Blsrepresonts motivation. As a result, many practicing 
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tcaclicfs do not understano, nov do they aiinieciate tlie signi<:icanco 
of mqtivatton. 

Vfhat the lesson plan calls "moti.vati.on" is ren] ly atLeni’.on- 
gecting, ACLeutlon-gettlng is vse£\ I i-;;Lcrnal devJ.cti such as 
yelling, or telling a hunoi'ous story, or "introducing" a subject. 
Ifotlvatidti, on the other hand, is v.iiat a learner feels vrhen he tc- 
fyonds to a stiw.ilus and Unov:S/'..jea^:f_ntj:;l v.'helhcr or not his roEponsn 
Is adcquaia. If he discovers the respous'e is adequate, he feels 
good, lie call it pasit:^^ rev : d. If he discovui u LtiC 
response is Inadequate, he rc.cclvuc (or feels) nega.ti):S. 
reinforce;.;ent. That roinforccir.f'.'.t sysfcr.i in a High. InUevisity 
Learning Center Is translated into rcui'ci da , oi' points, or sitiiple 
charts, graphs or records. The formal or liiformnl reinforceiiic-.nt 
syctem sets an "affective tone" vlii.cli is aiotivction. In other 
word; , motivation is t!vs re sul t of a learner cnperlgnoing the 
coatlnRenclos , or pay offs , reE-.'.lts of hj a beh aviors . 

This is a fact of life intritinic to all hobavior, hut it 
is rarely used In Che classroom or clinic as a management lochniqua 
in learning to read. 

An understanding of the nature and slgnifleauce of motivation 
requires tte entire ins tructioiial system to bo modified to present 
each stimulus or sat of stimuli to each learner Iraned lately following 
each learner's activa, overt rosponse to a previous stimulus, Thxn 
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is inteusive li^arning is different from wViat is allowed to occur 
In the traditional, teachar-diree ted large or small grouts lesson. 




c. Immediate feedback Is provi ded f or each response . Motiva- 
tion managemon t depends upon the Icarnor kiiowin Imnied lately the 
appropriateness of his response. Hov; else can the learner experie’ice 
positive or negative reinforcement mIucU becomes the impetus to 
attend to the next stimulus? 

d. Whatever Is to he learned shou ld be a rcinrorc.ed 'rewarde d) 
response to an appropriate stimul us or sot of s timid 1. Toaclil.ng is 
behavior management. The teacher structures an enviroiment In whit' 

prescribed stimuli are presented to the loaruer, and the learner's 
respor.sos are appropriately rciiucrccci. Too often, tojiohors 

expoBe. children to infoimatlon and assume tl’.au infoir.atlon is 
learned. Eut Intensified ler.rnJ.ns is niOre than s’cre exposure. 
Learning occurs i.’licii tha learner overtly rosponds, and that 
response is iimsadiately reinforced posit J.voly or negatively. Mere 
linowledge of the response's adequacy or* inadequacy is usually 
reinforcement for most noriiiel children. Tlie Important point is 
that unleis an overt responso to prograi-'ined stiTnull is reinforced, 
the teacher cannot assume Icarnins to have tahen plaec. 

D. Peraonnel - Selection Staffing and Training 

Applying the principles of Icarniiig described requires a 
very different teseher thati trsditionnlly sought i.t\ retnedtal 
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work, 'i'he teachor must be a wariaj,ei;, n.?t a diaiieDKor of visdoiu. 

The cianagei: dofoi'intues tlio lergot behaviors to be achieved, I'he 
manager dlagnones. He programs tl;e learvier accoi'dltig to the SO 
IHAT Diagnosis and then matclics sets of hard and software to Llia 
content need and levels of cncli Ic.-.rnor, Tiie instruclion becomes 
largely self instroetion or snail pupil‘'ter,m loai-ning. The managor 
or teacher adjusts resources using the Taxe ly (Figure A) as a 
guide. The Taxonomy offers eight varir/clca the teacher can manipu- 
late until he gets the desired response from the learner. The 
learner controls his own rate of learning a-,id aliaost all Uie record 
keeping. 

Examples of types of materials to ter.cli rending used in this 
type of "learning center" appear rjltli the interim, objectives in the 
APPENDIX, The teacher's job is to n<anlpulate these materials in 
relation to pupil needs, to modify materials so that they can be UBad 
as self-directing and self-correctlns Icartllhg experiences, and to 
give on-the-spot first aid to the learner wlien the materials to teach 
a specific behavior are unavailable or non-axistant . 

Teaching staff for i-.e project vjill be selected frorr fclie pool 
of personnel identified by each of the school districts. We. will 
welcome recoaunendstlons and advice froai the Icadershi.p in each of the 
Sevan school districts on candidates for participation In the project 
To the eKtent that para profc».*;iounl8 may be used they will be 
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selected frora tlij neds^d'orliecidr. ol: tVio tJirj;et pop.. ie t. Ions . 

The criteria \;c apply in Iho selectien of ceuc'idnlcs 

will include a record of succecs or demonatrated ability to 

• relate to and interact pr rductivcly \-lth young 
people of tlie target population 

• apply current ki!Ov;ledge of how young people learn 

• respond flexibly to the changing needs of a unique 
situation 

• use learning materials from the area of reading 
and language arts 



• use diffe ent learning strategics in accord v;itU 
the needs of tne learner 

• work effectively with Individual students 

The nilninium qualification for a professional to be considered 
would be cortlflcation as a teacher by the State of Virginia, We 
reserve the right to reject recomaended candldntos In the event they 
fall to meet our criteria. 

We will select one professional to staff each High Intensity 
learning Center (IIILC) in each of the school districts. These staff 
personnel will work directly with the 123 students attending tlie KILC. 
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In addition LR-i.-U£;i will Ciiploy end co;uit^'r-i;ttC! tv.'o poopln with 



axtensivo expcrloncc rincl training in ro.nJf ng ?ncl individi: J 

Icairttine tocImiqucF ns fieJt; supervisors niid prograia monitors. 

To the extent fenstblo we vril] ntcerapt to so] tel tl'.sse sup trvisot.-s 
from those rccoru uonclocl by the school d j o tr l.c Ls . However, wo will 
seek enndidates from wbr tc-vor sources seem most likci.y to yield 
people the characteristics we are sccUin:;. The supervisors 

«1'1 provide cupport and enidance to each oC 'he teachers opotatins 
the UlLC on an Itinerant basis* One of the supervisors will be 
headquartered in the western part of the state, probably in Oickenaon 
County, while the other will be base - in the eastern region possibly 
in Lunenburg County. They will apportion their time equally among 
each of the hILC's and schedule assistanre aecording to the needs 
of each center. The supervisors will undergo training under the 
direction o*' LKA-UEI personnel In advance of the training of operating 
staff of the HILC's, 

A one week Intensive Lraining program designed to prepare 
the teachers to effectively operate the HILC's will bo held in one 
of tha schools located most conveniently to the greatest number of 
participants. The teachers will he trained in tlie tecimlqueg of 
diagnosis, prescription, selection of materials for spcolCic needs, 
Bionltorlng student progress and oianagonrant procedures. AH of the 
relevant rasources of LRA-UEl will be available during this training 
period which will be held lu advance of the ppanirig of the centare for 




students. The training will take piece In a fully equipped IIILC with 
local students participating on a limited bafis for deriioas tration 
purposes. Vriitla the teachers are in training, materials and equipment 
will bs delivered to the schools in preparation for the cs Labltshmenl; 
of each of the HILC's in the seven school districts. All of the 
costs of this traintiig will bo borne by the contractor. 

The assistance provided by the ttro field supervisors will 
be auginented by LRA-UEI porsoiinel in site visits througliout the 
duration of the contract at the rate of the equivalent of six to 
seven wan days par month. Site visits will bo wore frequent during 
the start-vip period and throughout the early stages of the project, 

Ihs purpose of these visits is to 

. assure quali ty control of the program 

■ provide needed assistance to t .« fiald supervisors 
and center teachers 

• waxlnilze the effectiveness of the prograiii for each 
participating student in each of the districts 

LRA-UEl will offer on an optional basis the opportunity for 
additional teachers in each of the districts to be oriented and 
trained in HILG techniques. One month after the HILC'S are in operation 
an invitation will be extended to as many as six teachers to visit and 
observe the prograin in opera tloa. Subseq. ^ntly on a once a month 




basis for the reaiainder of the school year the teacher oparatlilg the 
center will offer an after-school hours in-service training program 
for tUes® teaci.ers. The six particix'Stint; teachers in each fGhool will 
be expected to volunteer thsir after-school time since there will be 
■iQ compensation by the contractor for this training. This experience, 
could, serve a number of valuable purposes, such ast 

• increasing understanding and accep''mico of 
the HILC's by the other profc-ssionals in the 
school 

t facilitating the turnkey process at the beginning 
of the second year 

• iinprovins Che classroom teaching effectiveness 
of those partielpatlng 

E. Operation of the Centera 

During the first t,icoh follcv.-ing the ti-iiinlng program 
the center teacliors will focus on prcpr.rlng their centers to beconjs 
operational the next v;ccU, Contractor consultauta and flold 
supervisors t?ill be on site during this i?aok. Ihitlng the SGcoi.d 
week students will begin to be phased into the centers and by 
the third week all Of the students to be served Jr. the school will 
be scheduled, and vjlll begin regular attendai'ice , 



Students will attend the centers daily for tnscruetton In 
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It enables both student and tfeacher to dcLorininc expeditiously 
the objectives coijiplo ted to critcrio;) level, elnpscd titio, and 
materials vised in working on tb’ attaiitnint of the objective, A 
description of this ICP (Inventory for Continuous Proereas) used 
for intormadiate matlicmaticc is ineludod in the APPENDIX , The 
system used In reading V7ill be simitar to that in mcthcTnatics . 

Additionally, students will graph their progress on appropriete 
vocabulary, reading cornprehonsion, stujy skills and associated 
reading skills for a minimum of 150 days. Individual testing and 
treatment schedules will be deterr, lined for each school in the bast 
Interests of the student and the school program. At the beginuiug of 
the program, students will be grouped and/or individually tested end 
objectives will be outlined for each student. These will be trans- 
lated Into individual prescriptions for each studenC. Approxiniately 
20-2S students will be seheduled into the center each hour of the 
day In order to properly accO!”modate the 125 suvidents in each school. 
Students will attend during regular scheduled hours. Students will 
also be able to earn extra time in the centers, as veil, upon achieve- 
nent of agreed upon objectives withlti Epeelfied time limits, 

Indlvldvial records of student progress and achievenenj. will 
be maintained by use of a self adpjinlstei-ing syKterj developed by Lft', 
which is based on the Royal McBcc card sort system. Tills systvin 
facilitates accurate recorvi keeping oud monitori.ns by the teacher. 
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forma that v:iil be provided as a motivating device toward the ful- 
fillment of the Individual contract they have developed with the 
center teacher. This technique has been used with a high degree 
of success in previous HILO projects. 

Students will be expected to sign in and out for each session 
in the HILC, Attendance records will be reviewed continuously as an 
aid in identifying students who may be lagging in progress. Time 
will be scheduled o-n an individual basis during the regular day for 
students to ''make-up" lost tine due to absence or other reasons. If 
t* becOTies necessary after school hour sessions may be held in 
schools where It is feasible and bus schedules permit. 



F , Objectives, Guarantees and Cost Implications 

Two Xevftis of objectives form the basis of ths coiiL-nanications 
program and relate to the guarantees of ac'.5QVc:isent. 



Tarr.tnnl Objfcctive a 

The terminal objectives arc to bs msasured by estiibllshad 



Etandardlzed tests to be selected by agreement between the contractor 
and the school dir trict project managers. We reeoinmend tha use of the 
Stanford Achievement Test or the Califarnta RoadJ.ng Test*. The two 
major objectives are broadly defined as Reading Comprehension and Total 
Reading as Indicated by the achievement test scores. The base fiat fee 
for the delivery of these objectives in accord with the conditions 
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described below is 75% of tbe total average niaxitnuai allowable per student 

l.e. $63.75, 

Condition 1; the mean acbiovemene scores for 8071 of the pupils v;ith 
90 IQ's or better will be on- year's ivicrcasc in the 
first six months. 

Condition 2s the mean aehieveinent setM'es for 15% of the pupils with 

90 IQ's or batter in one year will be 1,5 year's increase. 

Condltl. 3; for pupils in the 70 - 90 IQ raujic, the moan achieveinoiit 
scores will shot; one year's grov.’th in one year, 

Contingeneies; 

1 . For Cond ion 1, ebovc, a V anus of $4 v?ill be paid 
above tbe flat fee for each j.upil for each month 
gained above that indicated, 

2. A deduction of $4 will be made from tbe basic fee 
under Condition 1 for each pupil for each month 
below the base of one year gain in six monthfi . 

3. A bonus of $4 \,’ill be paid above tlic flat per 
pupil »or every month gain above 1.5 years gain 
in one school year under Condition 2, 



Interim OhlGctives 



Ineluded In the APPEKDIX arc several of the interim behaviorol 



objectives end types of materials to be used In tbe High Intenaity 
learning Canters coded to the Ta-.-onomy of Instructional Trcatmente 
{Figure A), These objt.cttvcs are samples taken from our coniprehcnsiva 
banks of several hundred objectives in reading dovoloped and compiled 
by LEA-UEl personnel. Trie specific list of intoritn objectives to ba 
used is subject to agreement as a result of consultation with each 
school system's project manager. The contrector guarantees cacli pupil 
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vlll deinoiistrate proficiency on a battery of objectives selected 
Individually for each pupil based on Iiis in-take diagnosis for a flat 
fee of $21.25. 



Contlnsenctes ; 



1, For each objective over the basic battery In 
which each pupil demonstrates proficiency, the 
contractor will be awarded a bonus of S2 over 
the flat fee of $21,25. 

2. For every objcctivo under the baste fcattcry in 
which the pupil does not deinonstraue prof- iency, 
a penalty of $2 will be deducted.- fpopt the -at 



Testiog Conditions 



The contractor will present to eac,ii project manager three criterion 
test items related to the perforiiiance objectives (both interim and 
terminal) for each test item to be used. Testing on these items is to 
be admintstercci every sij; weeks after the initial "start up" period. 
Certain conditions relate to the use of the a tamlnrdlKetl tests. 
These include; 

1, The same test pre must ta given post. 

2, The sai level tect given pro must he given 
post. For eKamplc:, a pupil p-re-~testc>d o'.i tl'c 
California Readlnj; Test Junlo:. lli.ah Level cannot 
be poet-tested on n California High SeTiool Lc-vol 
because different levels of she. "name" tost lire 



not stat; st'lcalX v liori'-.'; ''' on _n con t I ttui-t' -'-; , llicy 
are In effect diffci'cnt tests, 

3, Because v;e are doolins with an cxtrcwc end of the 
curve, pre-tests must be gi’-cn on that level of 
the test at which the pupil scores near the mean 
Given th(= level of achievement of the target popu- 
lation, this will usually require a level of test 
that was noiniod for younjisters with chronolosical 
ages below that of the tai'get population. This 
means that the absolu te valuofi of the scores are 
Incorrect, But for the pux'poso of this contract 
the standntdiaed tests are criteria for awarding 
achiiCvement contingeneles based on grovjth, and for 
this purpose the pre-post growth scores will be 
accurate. To violate this condition is, on one 
hand, to penalise the contractor when extremely 
low achievers gain but do not show on a poot-test 
at a. higher grade level and on the other hand to 
penallsse the Virginia schools by awarding the con- 
Eractor contlngciiclcs for siinplc ECatistical 
regression to the nvaan. 



It should be reesriphasized that the r.us.’vcv of Btudents involved 
in this project in each of the schools is to© low’ to perurt a high 
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level of cost effectiveness. If there i.'cii'e 250 siut'cuUs p;nrt.j.cl[>D 
at each school It would be feasible to provide the siin’.i services and 
guarantee similar levels of achiever, i. at nt about one and c. half times 
the quoted cost for the present project. It should be noted, further, 
that the costs of program operation in the secovid and subsequent years 
will be signif'cantly bolov/ that of the first year since most of the 
materials have been designed' to be non- consumable with a life expectancy 
of 3 to 5 years. In addition, start up coots after the first year will 
be minimal. Thus a very high level of cost effectiveness in the tutiikev 
process can be guaranteed. The specific level will be do' lin-ed within 
the first month after completion of this contract. 



G. Facll j Lina 

This project will ruqui.rt- a ii:intr.um of one standard si 2 ,e 
classroom at each of the ei£litcen schools, VJitli the exception of 
the Jacox Junior HigVi School In IJorfolk nil school districts hnve 
indlcatad their ability to provide the required rp>.'icc, A portable 
classrooiu or other suitable space vrill be provided by tlie coiiti'cctor 
for the students at Jacox, Approxirnatoly 25 square feet per student 
is the standard to be applied In deetding on roc-n capacity. 

The clasaroems to ha used for this project vjill be renewed to 
providn an atmosphere in marhed contrast to the usual schcol room. 

We will make maxlra-am use of available room furnishings w'ithin this 
concept, Each room will be freshly painted. If nccenr-^ry, area c.'rpcts 
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installed, air conditioned, and decorated with window draperies, r.j’.ch 
room will be outfitted v.'itb severe! comfortable lounge-type ebairs, 
office type desks, and conference tables, study carrels and bookcases 
and file cabinets for display and storage of student learnins materials . 
IJooms VJill be equipped to accoranodete approKimatcly 25 students, 

Kach room will require a minimum of five duplex electrical out- 
lets to properly support the A-V equipment which will be used by the 
students. 

Audio equipment viill be used with headsets to minimize distur- 
bances to ether students while the carrels will bo equipped with screens 
for comfortable viewing. 

H. Iieariiinp I 'ate r lals 

The raaterlrls used in each IlILC include large varietties of 
BoftvJare and soriie hardware currently available from dozens of publir.Iiers , 
manufacturars and soma such as TPI are the result of national eurrlculutn 
studies, Ko oua publisher produces the entire system of materials at 
this time. Among the inftterials included in die av: :-, of eor.TiunicE-tlons 
are; Michigan Language Program (LEA), Barnell Loft Specific Skill Series, 
SEA Labs, Eandpm House Reading ProRram Programmsd Book Conference, 
Follatt Social Studies, Random House Skilpacers, Doleh Word Analysis Card 
Addisop Wesley Eeadlng Development Kit, 

The essential characteristic baste to all the materials is that 



each has been selected on the basis of at least tliree criteria! 

.1. The ntaterial Is designed (or easily modiiiable) to 
allow the content, level and rate of learning to be 
adjusted to each pupil with a 30--1 pupil-toacher ratio. 

2, The content is relevant first to the needs and secondarily 
to the Interests of the pupils to be served, 

3. Most of the material is self-directing and self- 
correcting. 

This l-'it characteristic is of slgi.if icance since it provides 
littnedlatc feedback to the Icatner, Feedback has been shown to have 
a positive impact on the motivation of the learner for future Involvt:- 
Biont in learning activities. Tliis factor coupled with carefully selected 
and prescftbed learning materials increases tbs student's cha.neo for 
Eucccssful ochievement and frequently leads to increased laotlvation for 
furthor Icarnliig, 

TUe materials arc ceded to EHlccled crli ifa ouch cm:. ekllJ.j 
level, mode of % rcsentaticn, learning iiiodo, |■.vCJltp sltte, madia and- co 
on. All of the materials arc ’-eyed to tlus r.t talnmant of EpEciflc 
performance" ohjectlveo in readj.ng. 

The materials have been tr.oUlfj.ed to that they arc non- consumable , 
Experlenca has indicated that they can be used for the ircifl part i;lthout 
eignificant replBccrnsnt costs for a pciic-d of 3 to 5 ycai'b. All of the 



required equipment and tnaterirls for the IIIIX v;ill be [■-•ovi.dcd by LR'\. 
The HILC offers the specific ccnbitiation of iiisterisls in reading: and 
language arts in the appropriate quantities needed to inieneify ir.struc- 
tion for the target population of the eighteen Virginia schools. 



Can high intensity instruction V7orl; in centers or in regular 
classrooms with 30-to-one pupil-teacher ratios? The system described 
here was designed for inner city ghetto classroows serving severely 
retarded readers. Varisitions of High Intensity Learning Centers have 
been used iu such classrooms all over the country x.'ith ci:ccllont 
results. To satisfy skeptical conservatives, the syste.in was derjonatrated 
successfully vlth delinquent boys who were severely retarded in reading. 
The system was also demonstrated for beginning reading In grades one 
and two where, until r'ccently, self directing riaterials were unavailable. 
The system has operated successfully in junior high schools in 
Philadelphia, and New York, at Roosevelt High School in the Dronx, 
Hestlngboute Vocational High School in Brooklyn and in nlementacy echools 
in Bedford Stuyvesant, Lower East Side and Harlem. An excellent system- 
wide application is now underway in Ossining, New York under Mr, Frank 
Crawford, ouu of Dr. Cohen's graduate etudonts, Flguro E la a sunm'.ary 
eff results generated by some of these projects. 

By definition, intensifying instruction prevents dyspedagogla. 



I, Results of Experience v.*ith. HTLC's 
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The data available froa the above projects indicate this. For children 
already victiria of dyspadngogia, intensified instruction is, thcreforcj 
a prevention as v:ell as a treatment, 

III , pROj'i'CT ofg^v it zattok 

A. Frolect Tet .Ti 

UijS project terni'.j viCh LRA aocuiuins overall ruspor.sibility, 
has been specir.lly selected to meet the required ditioiplineo and 
experience necessary for the succens of a project of this scops 
and nature. Total experience of the project staff incliidvs 
cuiTiculuai dfsvelopirtentj instructional desigUj reeding instruction, 
evaluation design and Implementation comrrjnity rolatioiif. , learnino 
center operation and other relevant experience in schools, rniiversitics , 
industry and job corps programs. i 

This chapter describes the project organization and managewent, 
highlights the qualifications of key personnel and describes infonna*- 
tion exchange and quality control procedures. Complete reeumss of 
Staff tnetTbers are included In Chapter V on Corporate Bfcckgrouud and 
Experience , 

The organization of the project team is shown in Figure F. 

I£A will have overall responsibility for tho managenivint, super- 
vision, and coordination of the project and will work closely with 
each of the Virginia echool districts and the State Departmant of 
Education. Urban Ed, Inc. will be responsible jointly v?lth LRA for 
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Figure F 
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Dr. Roy Stern (LRA) 
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Urban hd . Inc 


Paula Barnak 
Lee Brown 
F, Colt Butler 
Dr, Joseph Lipson 
Dolores Moore 
Dr, Donald p. Smith 
Diana Smith 
Sif Hlkstcn 




Dr, S. Alan Cohen 
Anne liictcr 
Dr. Made Ion Stent 



the training of the staff, establishment of the HILC's end the cqn- 
tlnuons monitoring of the program throughout the year. The follOMlng 
paragraphs present brief vitae of key pessonnel. 

Dr, Roy Stern of LRA, • curTieulun developer and anrainistrator 
vitfi a Ph.D, from Hew York University, will be tlie project director. 
Dr. Stern has extensive relevant exporlencc, h-avlng served with Book, 
Allen and Haniilton, Inc., reanasestent consultants, and having trained 
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sciiool pC!j.-sonut;l i,n pvgjr.ct: rniiiiif -caU i;cclu'.i(iues . Ha has 
CKtcnsive cv-pario’iica In the iiut;.v;.<Ui;ili:.aLioii of instruction 
in tcachor Ci’clnintif Dr, Stem \;11}, provide loadership 
for all project p; 'I'sounol , His 20 years of CKperionce in otUica- 
tion is directly applicable to this p’ojec.t. 

Dr. Jose.h Liaison of LP/,, a physicist by training x.’ith a 
Ph.D, from Univei'siiy of California at borheley is a curriculum 
designer with considerable citparievice in development of materials • 
for Individualtiied ins true uion. Hu was responsible for the develop- 
ment of the IPX mathematics t.aterinls at the learning Research and 
Development Center at the University o£ Pittsburgh. Dr, Ltpson 
^coently led a team of consultants in the design of a completely 
innovative sehool to be established in the WashlngEon, D. C. area. 

He Is a natlonolly respected leader in individualization of inatme- 
tioii, che use of performance objectives and the developnent of ertterton 
i terns , Dr. T-ipson lies influenced the development of many teachers 
and administrators through training programs In which he has partlelpated 
Dr, Lipsen will be involved In. the eFtabliahment of the learning centers 
and la the monitoring of program effectiveness. 

Dr, Alan Cohen, Vice Jreoident of Urban Ed Inc. and Frofar.sor 
and Director of the Reading and Language Arts Center of the Graduata 
School of Yeshlva University w’ill bear major reapetisibllty for 
establishing the high intensity learntns centers and training the 
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operatltis staEiT, Dr. CoTicn vras oaa of the founders of the war oi\ 
povei'ty program, MobilisaLion for Youiih. .He lias concentrated on research, 
materials and system dcvolopment, writing and teaching in two major 
areas; teaching reading to the disadvantaged and learning 
disabilities . 

Dr, Donald P, Sraltli, carronlly prefecsor at UivivciT.lty 
of Nichlgan and author of lJt\'s tiichigsn Lr.iignngo Prograiv,, will 
serve as a consultant on this project. Dr. Smith is a leader 
in the field of progi-ammed instruction and classroom managciv.C'nt 
particularly as these areas relate to reading instruction. He 
has .served as a consultant for several national and state 
ersanizations and many local school districts. He Is responsible 
for the training of a large number of classroom teachers who have 
demons trated their effectiveness as instructional mr.nogers of 
the learning process In reading. 



B> Gontractor-Sehool System Information Exchange 
Froeaduras 



The procedures for assuring proper inforiLurtloa exchange 
Include pariodlc reports, meetings wlLh and vlsiuetion by non- 
participating Virginia school district parsctmiol end a project 
newsletter. 



Periodic Reports 

These will he made to the project mfinagor every six weeks 
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or other periods to gctioraHy coinelcle vith tho ri’pctClng periods 
of the seven school districts, Tnese rcpoi'ts wrll include; 



• An evaluation of the ptcgrara of the prt'ceelins pariod 
dealing vith (1) the objectives specified and the extent 
to which they V7ere attained, <?.) student grorfth in 
reading achievewont, (3) special methodology or tech.nlciucs 
used, (4) problems, (5) relations v.'itti scliool personnel 
particularly regarding help received in dealing v/ith 
problems , 

• A plan for the following period to include (1) objectives 
set for the period ahead, (2) rc.tliodr, to be used in 
reaching tlie objectives, (3) any cli-.nge in schedule of 
students, (4) anticipated problems, (5) h. Ip that might 
be needed from the school system. 



Three times during the project at each school, the center 
teacher and field supervisor will conduct iriectlngs for regular 
school distrleU personnel. The maeUlngs t-JlIl be designed 
to Inforin teachers on what wc arc doing and how \m are doing 
it. The entire staff at each school will be i.nvlted to attend 
these meetings. Participants would have tlio chance dm’ in g 'the 
second half of each tvo hour meeting to separate into small 



tectings^With Teachers and Ad niinistr.-.tms 
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giroups to discuss in greeter dutail. nay asi.ccts oi the project 
o£ special interest to therju Tills ;7ill ba an opportunity for 
the professional staff to fecdbacU into the system as v’ell as 
for them to become better itifcrtatd on tiie project’s operation. 

Similar woetlngs hill be held with building administrators, 
fidministrative role and Interaction v;i*th the project will be stressed 
in these meeting? along with tlie informatJ.oii eKchange. Among the 
major purposes of these meetings arc the smooth interrelation of 
the project with the on-going school activities and preparation 
of personnel for the turnkey process wlileh is to follow. 

Personnel who hav\, participated in these meetings will be 
encouraged to visit and observe the program in action, Adiuints trators 
will be requested to cooperate in making this possible for their 
teachers. These activities will be in addition to the oputovuil 
training program previously described for a limited number of 
teachers in the section on Personnel in Chapter II. 

AiS previously indicated we highly rcconimcnd slriill.nr meetings 
for parents. The contractors will be pleased to cooparate with the 
schools in participating In this effort, 

Proiect RevTslettcr 

Three tircss during the period of this project the contractor 

will prepare a newsletter fox- disfrllnitlon to school personnel 
and coiiimunlty leaders. Tlie newsletter xxtll be designed to 
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brontUy ti;s reao’ei'S o£ tlis iint.nro of tlic jJX'ojoctj the people 

involved, the participants, the prot.r&sfs and invite incjulry, feed- 
back and visitation. 



C , C ontractor Evaluation and Quality Control ProBrams 

t'valuati.on and quality control t:ill be provided through 
continuous monitoring avid supervision of tie project aiid project 
oparation staff by senior members of the contractor's or£nnikatioa, 
Tlie on-site supervisor will monitor program and porsoi.ucl 
on a regular basis while senior slaff of hlhV, and Urban Kd liic 
will do so on a monthly Ccheclule,, Keviow will consist of: 



• evaluation of individual and group progress 
against the ohjecti.vss. 

• Identification of students progressing at levels in 
discord with expsctancy. 

• planning of techniques to deal with contingencies 
and particularly V7lth the special needs of individual 
participants . 

• observation of project operation staff in performance 
of their functions. 

Elis type of review is atfnad at tnalntainlng and upgrading the 
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quality of the project through roconsisridetions for noc-J.'cl r.iOvh'fi.ca- 
tious. Senior staff members of U'.ie contractors will discuss aity 
major clicnges recomitencied with the project manescr to assure, agreemont 
and support. 

IV. PROJECT MAKWISG SPECIPICATIQHS 

liie manpower requircticiits arc dericribed in this chapter^ 
iii relation to the schedule for "start uc" activities end the 
continulug activities for th>e duration of the project, 

"Start Up" /. ct ivi tiep 

We will be prepared to initiate project activities within 
two to three VJeeks after the contract xn awarded. 

The proposed sequence of activities and manpovjer needs 
relevant to the early stages of the project follow. LRA-UEl 
estlniated manpower requirenierits are indicated in parentheses in 
terofi of man-day equivalents. Figure G sutnrrtartEBS the tashs, and 
the approximate schedule. 



Tasl, 




1. Interview and soloct eighteen project staff members 
, to serve as center teachers and two field super- 
visors from pools provided by the seven districts 
(12 - 14 man-days) 



2 weeks 



2, Orientation and training of two riexa supervisors 
(5 Bian-days) 



I week 
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3. Intensive training of tescticrs to o[>erate HILC*s 
(10 man-days) 



1 week 



4, Set np and equip High Intensity Learning Centers 
at each school 
(10 tian-days) 



2 weeks 



5. Students begin to be phased into lIILC's and are 
tested, their needs diagnosed and r.-.atcrials pre- 
scribed 

(10 - 14 man-days) 



2 weeks 



6. Concentrated assistance and support for tenclicrs 
at each HILC by field supervisors and LkiV-UEI 
personnel 
(20 roan-days) 



1 week 



2, All par ticipatins atudents report regularly to 
lIlLC's 



kt-mr.irdc'r o 
school year 



The scht'.dvilc it designed so that at tli.-'. lieginniiig of 
the fourth weeds of the-, project romo students will begin truc- 
tion ip the lIILC's, By Che end of the sixlli wee-k all sludaurs 
will be reporting on a regular basis to the llILC's for reading 
instruction. 

As indicated in Figure C several different ectlvities will 
be occurring simultaneously. Fo?: eiLample, while iiitcrvlewB are 
being held with teacher candidates the field supervisors will be 
involved in orientation and training. While Interviewing of 
staff and training are taklns place materials and equipment will 
be delivered to each school and the HILC' a will be "set up" and 
readlea for operation. 

It is crucial to the cost effectiveness of this project that 
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students be seh«dulcd in any given school to jirovide a cvitleal 
mass, within the existinc constraints of the KFP. Thus, schodullng 
of studants should be done so Chat approximately 25 students are 
scheduled for the HT.LC each hour of the school day. This will enabl 
Che center to properly serve the 125 students during the five to 
si;£ working hours of the usual school day as wall as allow tima for 
movemanC of students, malce-up work due to absence and incentive time 
for those progressing well and desiring additional opportunity. 

B. Continuing Fro.lect Activities 

The conti! . 'hig act ivi'. If S include £;11 ths aspects of 
individualized Instruction vzila llio participating studonts, 
coordination, supervision and linisou with the center staff 
and local school personuol avid cvnluatlon • vid quality control. 

Ihougli the cetrCor Lcachciis will coutlauu to be ciUployees of the 
local district and contlnne to bo conr.idcred as such they v;ill 
be expaeted to take, their direction relevant to center operations 
from the project field supervisoi'S and/or LliA-inJl parsonncl, 

Every effort will be made on tbe part of the contractor to assure 
harmony with the on-going school operations and personnol. potential 
problem areas will be thoroughly discussed well In advance with the 
local school district's project manager to assure a high level of 
correlation of activities beft/een school and project operation. 

Aftev the first six v/eeks of project activity the field 
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Bu^>ervisovs v?lll begin a regular scliedule of visits to eacb of the 
centers. The proposed manpower allocation for this project will 
enable them to visit each center for on'j day p.pproxinistely once 
every other week. They will be on call t- respond to special needs 
of any center on short notice. In addition LR/l-UEI personnel will 
make site visits throughout the year. These will be more freciucnt 
during the first few months of the project and gEnerally decrease 
In number as the project progresses. LRA.-UEI personnel will make 
field visits to the centers equivalent to six to seven man-days 
per month over the pcri.od of the contract which is estimated at 
approximately eight and a half months. During this time the ThA-Ulil 
personr-el will evaluate, woi'.itor , assujt quality control, 
participate in informaLlon cxcliaagc and training mEt ti.ng.'. 

With opevntioiiaT itaff ar.d %.-l,th teacher and ecT.’inis trati.Vv 
rcpimseiitatlves of oach of tljc schoC'ls. 

Part-time clerical assiKirmcc is expected to ho proVidad 
to each HILC by the local school, 

V. eOBPORATE BACKGROUND 

pach of the two sections of this chapter dcscriliss the 
background, objectives aiid prevtoua- exparicitco of each of the 
participating organizations of tbe consortium and IncludGS 
resuines of the key personnel. 
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Learnins Reacarcli Assoc. Infos , I.nc. was orijaniKcd by bee 
D. Brown In lytiS as a subsidiary of UuiversaJ, Education Corporation, 
New York, Now York. Its' object i.ves' are two-fold: 

1. Ttie desis’'» dcvelopn’.ont, validation, production and 
professional dissemiviatiou of innovative learning and 
teaching itia ter inis for e.JoMontary and secondary 
schools . 

2. The provision of consulting services and in-service 
training progtasis to schools, education-related 
agencies, and other youth-serving organisations. 

In the area of materials development, LRA's Initial fields of 
concentration arc conununication shills with omphasis on reading, 
mathematics, science, and social science. Its learning and teaching 
iiiaterials have the following cl iiracceristlcs: 

• Developnwsnt of specific student pcrformcnco objectives 
and porforra-aiice tas5rs for raaximuia student and teacher 
feedback, and Cor practical, effccClve evaluation, 

Ewpliasis on active Btv.dcnt Involverisnt with the 

■ •'stu.ff" of learning, with conco.aitant empliasts on 
active .■.achcr involvcnisitt throuGli diagnosis and 
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prescr j.ptJ.i'.'i' . 

• Ikixlmuiii 'lil.lty oJ! tcnclisi’ ai'i'J Icr.ruci" ctrfitegies 

to achieve desired Etv.doit pcrforii'raice . 

• Teacher training and the training or highly proiesoioutl 
consulting and i\i-';dcc, Ling personnel to iuiploircvit the 
educational design, 

• System analys-is and functional f.i.o’.i design in terms of 
tasks, production cormiDiianUs, and "closed loop" systciiis 
of quality control and checkout, 

• Multi-media options that include a vide variety of 
relatively inexpensive equipment and material, including 
audio-visual and manipulative, 

• Emphasis on problem-solving as a learning strategy, 

• Etttphaais upon continuous student progress, 

, Fiald testing and validation of products In school settings 
during prototype and revised prototype design stases. 



LS&'s profcssloiial staff consists of siKteen full-ttmo 
people and twelve full-tlma and part-tiine consul tants, Ivi addition, 
it has direct accecn to twelve full-time profcsolonalB and eight 
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pnvt-tima conri.:1.t;!vii;L' MrJ Mii.ibcrr. of linj.voi:!:':’.! 1,'clu.cat'. loti 
Carporntiou'ti Ki-.t lo'.ic.l /.Jvisory Kcuvrd, Vi'ilbur 

Cplien, cuirrencly Dean oy. ut'e ScDool, oT tt’uc.at J.on, Tlic University 
oi Miclii^nij, and fotr.ier Secretary c " llealUi, i;di'cat;d.on i .ul UalCa.re; 
Robert Glaser, Director of Lcariiins Rcseare’n and De\v.lo;-.:;cnt Center, 
The Univeryity of Pitttburj.b; l-iartin PcutncU, Director of tiie 
Instituta for Developmental Studies, New Vorl: Undversity ; Layard 
Kustin, Executive Director of the A. Plniliji Randolph Institute; 
and Kyrtlc Kc.Graxj, Chainv.an of the DopevUm-ent of Developmental 
i'sycliology at Rriarcliff College. 

The professioaal staff iticludes three tiavTicmaticians, one 
physicist, two reading and tnslisii spsclalir.ts, one biologist, 
one educational psychologist a.nd one sociologist. Additionally, 
twelve staff aicnibers have had teaching and school adiai.nistrative 
experience, renging froni llriiited to extensive expsrienco at all 
grade levels. Finally, of the pr ‘'SEional staff, thirteen have 
inaster’s degreos and nine have doctciates. All profcsslonnl staff 
Biembers are familiar with, and siost have practical experience la 
the developmanC of parformance objectives and parforroance tasks 
based on observable, maasumblc student behavlore in basic skills 
and key coiioepts in all major currieuluni areas. Most professional 
start uambers are familiar with the techniques rard problems of 
systems nnalysts and maiiageinent, includlns planning, prcigrammlng 



i^3 



1 



and bud£ctf.ng systernsj Pi:ilT and Cjnt s/si-cv.\r, and charts. Since 
Loarning Rescavch Associnico , liic. ts less tJinn tv.’o ye rn old as 
an oraanii’.ation, its expsi'i cncc with rclctccl projects is li.v.n'.tcd , 
However, V.cy '■talE aionihcvr. v'lo woeld be cli.r£;ctiy involved in the 
Virginia Projec "Gac.rr.iiCce d Stud-. ui: Achitvomen!' in Koeuling 
I'hrough 0,. ration ol; A.cocXcratcd Ac'ii'--vr:iK'Ut Ci-i'toi-s" Imvo 
di'.-oot;Qd ai'.d/or had "lu'nds:. c.i" cr.prv ionco. in the Collowing 
related projects; 




Inctlvidual ly Proscribed 

I tis tructlon "~Tll*i)"- " ” ’ 

dal-.lcaf and Baldwin- 
Kliitehall School:-, 
Pittsburgh, Pennsylvania 



Nova Schools 
Florida 



Prescribed 




, Ft. Lauderdale 



A totally new sy.‘itcm involving 
diaguor.is oi student pcri'oUnance 
In elcMCiitary reathcmstics and ■ 
scj.onoi’., and prescr ipLivc 
activities and learning materials 
based oil student pctfon:iance 
objectives was developed, validsLcd, 
and installed at OaUleaf and 
Eaith-rin-I.'iiite’nsll schools under the 
auspices of the Learning Research 
and hevclopnient Center, University of 
Pittsburgh, sponsored by. the U.S, Offlcv 
of Education, 

An extension and improvcmcrit of the 
original IPI system which utilincs 
ler.rnlng activity packages (LAPS) . . 

L‘!’S consist of 20 to 100 pages 
rcproecntlng a private learning 
plan for the individual student end 
includes student study guides, 
suggested resources and self 
evaluation sections. InsCnictlonal 
goals are stated in behavioral 
terms at the beginning of each 
TAP and various ways for the 
student to reach the goals are 
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sugsestod* Developed for K-12 
li oil the mstjor subject tirctis. 


Inventory for Cont.lniious 
j*ro;;ress £tT Na Ties 
devefoped by LRA 


An liidividualiy.od instructional 
system us ins ^^^y sort cards, 
poiCoriiiS.nco objoccivos, criturion 
rcfci.i; need tests (jn-o & post), fu’d 
e wide, variety of Icarr.ins activit- 
ies and materials. It isself- 
diroctin^ t.nd sc] f-coi,‘re.ctin^' and 
provides tbo student and teacbor 
vith tontinuouo r.ioni torin:; and 
lift.’.edi.atc feedback. 


_t_;':iuer '.J onrJJ Syi, te-..- 
Fai't Lincolii-I.’ow To’..-n 
(District of Colu:.:bin 
Fubltc Schools) 


A Cl \p)c'te cducatiPiial system for 
tfie p’, *.»pC'Si*d n;-v.' t,y ot 

Fort l.incoln for an estinu'.tVd 
10,000 students, a''os 3 to If. 'ihn 
p]fi-.uiri- procciiE involved parents 
and r. tne'ents, and war brved on 
f.tudt.'it pe-rfi-. ,.'.anco oV' ject ivna and 
related porfciirinai'.ce t isks in all 
f.i.ajor uurricului-,1 areas, spoeifically 
dc.veliipcd for the project. 


I’lf.unJng t! Complete 
K ducii Fi o n S'l“ S>’ si.” r 
Weltaie (Nc-.F 

York City) 


This current project invt.lvcs 
deni-ntn- a complete c.r'i' cat’ anal 
system for an cducati.on co;i;plcx ou 
V.'clfaic Island (in tlie East Kiver 
next to'Hanhattan) in Kavr Vork City. 
The. design is based on student 
performai -tc objectives and related 
performauee tasks in all major 
eurr'iculu.n areas. 


draining School Admin- 
istrators in Proicct 
Kaua^eaianu Tecliniquos 
(New jergey Urban 
Uevelopiaont Council) 


Adi.jiiilstrators wore trained in project 
tnanngc.Ticnt techniques for oCfoctJ.ve and 
efficient plaimlvis and' execution of 
educational programs. Each partici- 
pa.tlng school district developed a 
nctvrork plan (PERT or CE-I) for allcca- 
tlon of all rcsQurcot, a time se.he.dulc! 
and constraints. 



Troinl ns T eache rs a nd 
AdrolTilsCtayovs in tlio 
Devclopricnt of i’crfom- 
ance Objcc lives and 
Critc'ion Itoms for 
Elementary and Secondary 
Students (Esscxville- 
Hampton, Michigan Public 
Schools) 



District personnel oriented to^JQrd, 
and practicing individualized 
instruction, produced learning units 
and criterion items at their instruct- 
ional level in subject areas of their 
choice. Kdy aduiinistrators became 
leaders in disscMinaticn of the con- 
cept and practice of individua' ized 
Instruction in thcii: c-.'a district and 
in other" Michigan school systems. 




trai ning Teachers and 
Admin ist rators in Ind iv- 
id ualization of Instruction 
(LHA Institutes on riTcTl^'i 
dualizatlon of Instruction) 



Dos lftri an d R valu atlon of 
a Piojcct to Increanc; tlw 
Achlevev.iant of tsJ.nth Graders 
K’tarded Tliree to Six Years 
In Keadlng and itat1ieV.iatl.es 
(Morristown, Km; Jersey - 
Academic AdvaneOEiEnt Project) 



Currl cultra nc velot i'Tep t: and 
ProRroia O c eraCIo ii-jgjjS 
Project. (Chicago, YHCA and 
U, S. Department of Labor • 



Korkf4h.op-institut.es ropre.sent a 
model for Individunlizatlon of 
In.struction, Teachers and 
administrators participate for 
three ct.rys by use. of pvc and post 
testa, perfonnance objectives, 
prcpcrihc'd learning activities, 
adaptation and selection of 
curriculum matcrl.'ili; , Ihe 
cxiic-r j once is deElgncd to be 
prnctic.al and load to 
i ..iied i.nt c iinplcmoirtation 
in the school.s, 

A special progr- n currently 
lu prevross using lndlvldun.llK.::d 
tec’mlfuieE . and a ir.il ti-medi^i, 
apiiroach Integrating rending, 
iiiatl'osnatlqs, acieivip. niid social 
studies to motivfttQ students to 
greater achlovcmsnt and Improved 
attitude toward Xcariiing and 
Ecl ool , 

OrgaiiiKed and directed nationally 
Itnovai group of teachers and 
teaclier-trainere In the develop" 
Biant of reading and watlematiCB 
curricula for high bcUooI drop- 
outs in tlie city of Chicago, 







Eva Uiat:lon of Pupil ft-rso'-'-'fl 
ar j . A d m lTxls c ,^n, 

1 5.011 Progi.'a ns £or the Bureau 
o£ Education Professions 
Developtuent of the U. S. Office 
of Education (Washington, I),C.) 



Per formed supervisory and teaching 

func.tiouH for fi.olcl- testing and 

tcacUcv-trainins purpenes. Project 
pr-e-c'ated OEO and is still ccnsidsi'Qd 
one of the outotandiug programs in 
the U. S. in terms of drop-outs 
who re turned to sichool or secured 
gainful cmployinsni';. 

An evaluation, now in progress, of 
two prograias of the BliPD involving 
development of coinfirehcns j.vc pro- 
gram descriptions, statciient of 
progiram objectives in performance 
terms followed by a monitoring and 
impact evaluation. Program ascess- 
went and improvement are the prime 
goals of this project. 
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